-- Logs begin at Wed 2022-08-24 07:03:20 UTC, end at Wed 2022-08-24 08:18:05 UTC. -- Aug 24 07:34:48.070990 dv-cinder-854275-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 07:34:48.079391 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 07:34:48 2022] *** Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: nodename: dv-cinder-854275-1-iscsi Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: machine: x86_64 Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: clock source: unix Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: pcre jit disabled Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: detected number of CPU cores: 4 Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: current working directory: / Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: detected binary path: /usr/bin/uwsgi-core Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: your processes number limit is 47680 Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: your memory page size is 4096 bytes Aug 24 07:34:48.084804 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: detected max file descriptor number: 2048 Aug 24 07:34:48.085593 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: lock engine: pthread robust mutexes Aug 24 07:34:48.085593 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: thunder lock: enabled Aug 24 07:34:48.085593 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 07:34:48.085593 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 07:34:48.125867 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: Python main interpreter initialized at 0x5629e7f729c0 Aug 24 07:34:48.125867 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: python threads support enabled Aug 24 07:34:48.125867 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: your server socket listen backlog is limited to 100 connections Aug 24 07:34:48.125867 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: your mercy for graceful operations on workers is 90 seconds Aug 24 07:34:48.126232 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 07:34:48.126308 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: *** Operational MODE: preforking *** Aug 24 07:34:48.126558 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: *** uWSGI is running in multiple interpreter mode *** Aug 24 07:34:48.126558 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: spawned uWSGI master process (pid: 88223) Aug 24 07:34:48.126768 dv-cinder-854275-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 07:34:48.127859 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: spawned uWSGI worker 1 (pid: 88233, cores: 1) Aug 24 07:34:48.127859 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: spawned uWSGI worker 2 (pid: 88234, cores: 1) Aug 24 07:34:48.127859 dv-cinder-854275-1-iscsi devstack@keystone.service[88223]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 07:34:52.466930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:34:52.466930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: from cryptography.utils import int_from_bytes Aug 24 07:34:52.467670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:34:52.467670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: from cryptography.utils import int_from_bytes Aug 24 07:34:52.467670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:34:52.467670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: from cryptography.utils import int_from_bytes Aug 24 07:34:52.468188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 07:34:52.468188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: from cryptography.utils import int_from_bytes Aug 24 07:34:52.521677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:34:52.521868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 07:34:52.521960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:34:52.522059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 07:34:52.522142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] command line args: [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:34:52.522215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] command line args: [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 07:34:52.522382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:34:52.522472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 07:34:52.522472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ================================================================================ {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:34:52.522610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ================================================================================ {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 07:34:52.522724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] admin_token = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.522813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] admin_token = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.522971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] config_dir = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] config_dir = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] config_source = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] config_source = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] debug = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] debug = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.523980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.524935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_config_append = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_config_append = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_dir = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_dir = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_file = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_file = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.525945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.526989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.527820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.528976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.529983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.72/identity {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] publish_errors = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] publish_errors = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.530917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] standard_threads = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] standard_threads = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.531967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] use_journal = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] use_journal = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] use_json = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.532596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] use_json = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.535857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] use_stderr = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] use_stderr = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] use_syslog = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] use_syslog = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 07:34:52.536401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.536962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537467 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.537973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.538933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.539994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.540956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.541993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.542918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.543538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.544983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.545885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.546939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.547976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.548929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.549983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.550974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.551972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.552978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.553904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.554931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.555957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.556887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.557980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.external = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.external = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.558801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.password = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] auth.token = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.password = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559654 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.559774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] auth.token = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.560897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.561961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.562900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.563960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.564995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.565090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.570946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.571976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.572978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.573991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.574990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.575921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.576982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.577988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.578943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.579939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.580996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.581928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.582875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.583985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.584984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.585919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.586957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.587945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.588945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.589866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.590064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.590169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.590267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.590858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.590976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] role.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] role.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.591905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] role.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] role.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.592886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.593866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.594944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.595988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.596916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.597949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.598952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.599994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.600840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.602871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.602977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.603977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.connection = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.604947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.connection = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.605954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.606629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.606708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.606780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.606852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.606922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.607962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.608113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.608193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.608265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88234) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:34:52.608358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.608593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.608867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.609105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 07:34:52.609279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 07:34:52.637577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.637803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.638026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.638295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.638531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.638764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.646584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.647248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.675857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.676004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.676400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.676636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.738118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.738286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.790286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.790490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.790573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.790723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.948983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.949058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.949170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.949253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.949437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.972972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.973922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.974053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.974146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.974265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.974347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.974481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.986031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.986153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.998935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.999140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.999227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.999370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:52.999453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.004963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.005560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88234) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 07:34:53.021206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.021415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:34:53.021531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.021746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:34:53.021829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.021941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:34:53.022094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.022214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.022298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 07:34:53.022472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.022554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.022819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.022986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.023254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.023404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.023683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.033170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.033449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.033721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.033813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.034080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.034171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.034353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.034643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.041088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.041677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.044484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.044797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.045369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.045995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.046932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.047071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.047153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.047351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.047463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.047745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.048021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.071049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.071552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.072497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.072809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.073062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.073355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.078120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.078656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.078656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.081346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.081640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.081922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.086510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.086797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.088879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.089176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.091101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.091393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.091675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.092469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.092772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.093048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.093316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.093582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.094393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.094693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.094982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.095267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.095546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.095825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.096101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.096375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.110776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.111055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.113777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.114062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.115096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.117998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.118452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.119664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.121289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.122103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.125338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.126441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.128486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.129695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.131891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.132195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.132484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.132796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.133942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.134239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.134533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.149864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.150153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.150429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.150690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.151204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.151499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.151772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.152041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.157642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.159017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.159153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.159438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.160559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.160871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.162101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.162388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.162680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.162972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.163250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.163521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.163820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.164122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.164427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.164744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.174688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.174975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.175254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.175532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.175811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.176110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.176258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.176393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.176549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.176861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.177153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.177444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.177727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.178019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.192808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.195946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.197324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.198909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.201010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.205640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.205730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.205730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.205730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.205921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.205921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.207317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.207607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.207904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.209520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.210984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.213902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.214199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.214493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.214586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.214780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.215990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.218947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.219245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.219534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.219823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.222866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.223153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.223435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.227951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.228245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.228531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.230445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88234) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.230732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.231048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.231328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.231602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.231875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.232161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.232431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.232745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.233025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.233296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.233558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.233841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.234102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88234) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.235597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 07:34:53.235909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.236217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.236502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.236810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.237100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.237379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.237668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.237954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.238241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.238518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.238788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.239078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.239347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 07:34:53.270864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5629e7f729c0 pid: 88234 (default app) Aug 24 07:34:53.277813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c2b21-7c08-4622-a8e8-12db302cdcac None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:34:53.277933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5629e7f729c0 pid: 88233 (default app) Aug 24 07:34:53.278105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c2b21-7c08-4622-a8e8-12db302cdcac None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:34:53.278376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c2b21-7c08-4622-a8e8-12db302cdcac None None] PATH_INFO: `/v3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:34:53.282570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 791 bytes} [Wed Aug 24 07:34:53 2022] GET /identity/v3/ => generated 255 bytes in 11 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:00.834743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:00.835307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:00.835474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:00.839753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1/2] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:00.846308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:00.846724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:00.846852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:00.918238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: INFO dbcounter [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] Registered counter for database keystone Aug 24 07:35:00.938131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG oslo_db.sqlalchemy.engines [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88234) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:35:00.944473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writer thread running {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:35:01.132474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG passlib.handlers.bcrypt [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88234) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:35:01.132893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG passlib.handlers.bcrypt [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88234) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:35:01.196145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:01.197757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:01.331972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 2/3] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 07:35:01.816513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:01.885390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: INFO dbcounter [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] Registered counter for database keystone Aug 24 07:35:01.907677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG oslo_db.sqlalchemy.engines [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 07:35:01.913916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writer thread running {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 07:35:02.033465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a6818-1ac1-49ac-9a6e-c91134305d50 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:02.038043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:02.038679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:02.038963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:02.039240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:02.047342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:02.098156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING keystone.common.rbac_enforcer.enforcer [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:35:02.098455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cad7d-69cc-40c1-a5ac-ddd3db29a273 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:02.107822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 2/4] 10.222.0.72 () {58 vars in 1324 bytes} [Wed Aug 24 07:35:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:35:02.113821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:02.128726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d3d71-d6de-487b-98e0-8ae6360eeb76 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:02.133140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:02.134249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:02.134534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:02.134809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] PATH_INFO: `/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:02.136473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] RBAC: Authorizing `identity:get_project(project_id=8d87d471ae2b4849a0dda41c93ab2dfd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:02.188460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 07:35:02.188829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9742e2c6-2db3-4bdc-a36d-a6eca53e4ecc admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:02.193901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 3/5] 10.222.0.72 () {56 vars in 1207 bytes} [Wed Aug 24 07:35:02 2022] GET /identity/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:04.065435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a1e9d-8ba4-458f-a6d7-058f23105e0f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.065948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a1e9d-8ba4-458f-a6d7-058f23105e0f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.066114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a1e9d-8ba4-458f-a6d7-058f23105e0f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:04.070448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 3/6] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:04.076536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.076974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.077135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:04.097346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:04.098891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:04.106881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 4/7] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 07:35:04.374414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a1e9d-8ba4-458f-a6d7-058f23105e0f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:04.386855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a1e9d-8ba4-458f-a6d7-058f23105e0f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:04.391028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:04.391628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.391909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.392187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:04.399727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:04.401536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9b345-fcce-4a41-ab2c-0b5f86779ac5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:04.410728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 4/8] 10.222.0.72 () {58 vars in 1324 bytes} [Wed Aug 24 07:35:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:35:04.416275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:04.425999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850cd5d-8620-41e0-9fdd-7c287bc9b31c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:04.430368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:04.430996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:04.431279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:04.431557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:04.432900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:04.434989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600a14-1629-48d3-b29a-67ee45994e84 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:04.436527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 5/9] 10.222.0.72 () {56 vars in 1198 bytes} [Wed Aug 24 07:35:04 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:06.387100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2f77-ebbc-4870-8095-03a5a6dd0dd2 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:06.387655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2f77-ebbc-4870-8095-03a5a6dd0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:06.387655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd2f77-ebbc-4870-8095-03a5a6dd0dd2 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:06.389042 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 5/10] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:06.394237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:06.394663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:06.394789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:06.415027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:06.416200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:06.445039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978b54f-3925-4f03-a4fb-b802d0d1f5f5 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:06.445321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978b54f-3925-4f03-a4fb-b802d0d1f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:06.445607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978b54f-3925-4f03-a4fb-b802d0d1f5f5 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:06.447053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 6/11] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:06.452692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:06.452984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:06.453259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:06.494985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG passlib.handlers.bcrypt [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88233) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 07:35:06.495426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG passlib.handlers.bcrypt [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88233) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 07:35:06.497528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 6/12] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:06.541224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:06.542843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:06.642851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 7/13] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:06.764057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:06.788685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:06.869037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1297d-bb91-47f6-ad11-f43e34e75b30 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:06.903254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:06.903838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:06.904123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:06.904398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:06.912535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:06.915497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:06.925021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:06.925993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:06.926851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 7/14] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:06.931788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.012927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.015787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692773b7-f486-4419-a88c-306a4df1f86b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.018164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.032134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d903d-7875-42d0-8ee0-c3907596fa44 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.052526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.053613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.053893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.054167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.066762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.067374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.067681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.067951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.068615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:07.068993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.070437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.082640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 8/15] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:07.088117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.122624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.124183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.189684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00357a-30c3-4a03-9393-723cd5b2338d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.222835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 8/16] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:07.225067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.225707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.225986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.226264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.227505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.228451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.228982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.230581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 9/17] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:07 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:07.231294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.235870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.250332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.333771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bb186-4ae3-4cff-9823-56d9d2c35153 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.337083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8d6c2-9f12-43a4-b86c-17a529e80cdc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.368259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.369405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.369690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.369972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.371655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.372282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.372561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.372864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.373988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.375443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.377236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 10/18] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:07.382961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.390396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.391888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:07.410063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:07.411112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:07.411906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:07.411906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:07.411906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:07.411906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 9/19] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:07.416901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:07.465799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.473822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:07.484532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94618c72-5758-42c3-9355-331e807c1342 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.517382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f01fa-3916-4087-8b4c-bd54798a7eb9 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:07.520274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.520389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.520696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.520972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.521709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.domain.id=default, user.id=691d549b6a8240cfaab77208fe5f61bd) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:07.522015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.523458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd8470-a5f7-4d4a-9cad-42837c70e84d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.530157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 11/20] 10.222.0.72 () {56 vars in 1322 bytes} [Wed Aug 24 07:35:07 2022] GET /identity/v3/role_assignments?user.id=691d549b6a8240cfaab77208fe5f61bd&scope.domain.id=default&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:07.552122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:07.552787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:07.553073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:07.553373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:07.554045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:07.554352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:07.555806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce22105-fffc-4d17-9cbc-1ff7a2d737bf None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:07.577679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 10/21] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:09.531603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e83cd-6be3-4f69-a3e8-e4b73c4b749d None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:09.532130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e83cd-6be3-4f69-a3e8-e4b73c4b749d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:09.532130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e83cd-6be3-4f69-a3e8-e4b73c4b749d None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:09.533500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 12/22] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:09.538877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3a8-654a-499e-8a2e-5de9088897c3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:09.539706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3a8-654a-499e-8a2e-5de9088897c3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:09.540215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35a3a8-654a-499e-8a2e-5de9088897c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:09.561297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dd35a3a8-654a-499e-8a2e-5de9088897c3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:09.562839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dd35a3a8-654a-499e-8a2e-5de9088897c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:09.634862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 11/23] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:09.756979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211ffb6-f23a-49df-9c5b-a43bd2318969 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:09.757596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211ffb6-f23a-49df-9c5b-a43bd2318969 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:09.758067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211ffb6-f23a-49df-9c5b-a43bd2318969 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:09.759593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 13/24] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:09.765869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80c1e-b49e-49ef-848f-e706b401ee71 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:09.766402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80c1e-b49e-49ef-848f-e706b401ee71 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:09.766904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f80c1e-b49e-49ef-848f-e706b401ee71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:09.792031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-38f80c1e-b49e-49ef-848f-e706b401ee71 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:09.793646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-38f80c1e-b49e-49ef-848f-e706b401ee71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:09.850394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:09.875847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 12/25] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:10.015174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211ffb6-f23a-49df-9c5b-a43bd2318969 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.147235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211ffb6-f23a-49df-9c5b-a43bd2318969 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:10.168900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c64f-7261-4a71-ac1b-5afcabdc3c4b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.169189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c64f-7261-4a71-ac1b-5afcabdc3c4b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.169644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c64f-7261-4a71-ac1b-5afcabdc3c4b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.171419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 13/26] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:10.176895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.177424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.177901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.196052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:10.196718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.197001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.197269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.197674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:10.199690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:10.225675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:10.231092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:10.231997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:10.232896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:10.236962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 14/27] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:10.241586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.245735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:10.312416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 14/28] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:10.387977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.431038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e336-f0ed-4a2f-89c8-fd830c0c3a57 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:10.493274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:10.493914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.494189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.494466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.495281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:10.495763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:10.501527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:10.514805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 15/29] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:10.520645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.544008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07ccc8-4bc1-426d-a008-795797f7485c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:10.565102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:10.585028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:10.585785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.591945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.591945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.601354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:10.602968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:35:10.629614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:10.630503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:35:10.631698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:35:10.631698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:35:10.631698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:10.631698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 15/30] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:10.637160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.696970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:10.770290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1b7b-7121-493b-a0c5-c66512aa8adf None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:10.808154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abf8a2-87b7-412c-af9b-4f53b945bf42 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:10.835582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:10.836151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.836426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.836729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.841343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:10.843074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:10.844696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 16/31] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:10.846034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.847939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:10.848796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:10.849265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:10.849780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:10.850613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:10.851085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:10.852992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:10.867461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 16/32] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:10.868626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:10.890483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:10.990101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb0ce2-8d1e-41d4-8870-757ca9a88afd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:11.023619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183efee5-2449-43fe-a6dc-f063dfaf1df2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:11.035770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:11.036459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.036778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.037058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.038437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:11.040003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:11.041681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 17/33] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:11.042517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:11.070067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:11.071575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.072055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.072806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.074239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:11.076701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105655-b704-4246-bb1c-cc5e952d1aad None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:11.082368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.141147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.155970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b5d2-9495-429c-a366-904402dc3d25 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:11.193939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:11.194662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.195012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.195398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.196222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:11.197916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b085-4192-4c17-95aa-91ac332223ac None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:11.287732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 17/34] 10.222.0.72 () {60 vars in 1131 bytes} [Wed Aug 24 07:35:10 2022] POST /identity/v3/roles => generated 219 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:11.289504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38cabc-da79-460a-9a66-22b3c266219d None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.289954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38cabc-da79-460a-9a66-22b3c266219d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.290413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38cabc-da79-460a-9a66-22b3c266219d None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.292421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 18/35] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:11.293341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38cabc-da79-460a-9a66-22b3c266219d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:11.450955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38cabc-da79-460a-9a66-22b3c266219d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:11.471675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 18/36] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:35:11 2022] POST /identity/v3/projects => generated 301 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:11.473636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ce75-881f-4f4e-8812-11804c0bd2c0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.473925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ce75-881f-4f4e-8812-11804c0bd2c0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.474202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ce75-881f-4f4e-8812-11804c0bd2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.491112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:11.491817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.492117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.493069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] PATH_INFO: `/v3/domains/default/users/691d549b6a8240cfaab77208fe5f61bd/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.496913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=691d549b6a8240cfaab77208fe5f61bd, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:11.498950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9343-b01d-4a20-aace-17c0304f2765 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:11.503666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c9b9ce75-881f-4f4e-8812-11804c0bd2c0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:11.505276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c9b9ce75-881f-4f4e-8812-11804c0bd2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:11.584954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 19/37] 10.222.0.72 () {58 vars in 1361 bytes} [Wed Aug 24 07:35:11 2022] PUT /identity/v3/domains/default/users/691d549b6a8240cfaab77208fe5f61bd/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:11.586267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52854f3-29ab-4617-8585-a991287945ba None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.586557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52854f3-29ab-4617-8585-a991287945ba None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.586837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52854f3-29ab-4617-8585-a991287945ba None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.588427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 20/38] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:11.589984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc773-aff1-4057-99b2-86fb27a415c7 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.590271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc773-aff1-4057-99b2-86fb27a415c7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.590543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdc773-aff1-4057-99b2-86fb27a415c7 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.592004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 21/39] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:11.600606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.600955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.601185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.602520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 19/40] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:11.606317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:11.606969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:11.607473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:11.632527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:11.634281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:11.645726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:11.647464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:11.708554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.729523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:11.730935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 22/41] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:11.736243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 20/42] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:11.897447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:12.008044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e98741-4721-4e04-a763-d4aae0594962 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:12.012449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:12.047431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:12.048097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:12.048381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:12.048670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:12.049374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:12.050773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:12.115430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee44c4-4bec-4b99-a2e2-3911392d9efe None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:12.134438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 23/43] 10.222.0.72 () {60 vars in 1131 bytes} [Wed Aug 24 07:35:11 2022] POST /identity/v3/roles => generated 225 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:12.135251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:12.169471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:12.171259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:12.171733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:12.172187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:12.174396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:12.176519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:12.179690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 21/44] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:12.189910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:12.192614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:12.275705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:12.321822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b6555-5cf8-4d07-9fa3-cb35c18c41c5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:12.365835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:12.366031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:12.366281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:12.366545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:12.367118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:12.368563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b618fe-3316-4bc0-ad05-dedc42eeda73 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:12.368689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b112bb3-427e-4b7d-90a6-8bfe3c084b74 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:12.414648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:12.415271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:12.415568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:12.415825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:12.416501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:12.418787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1075dcf-345b-4b88-a200-c69b9356808d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:12.449310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 22/45] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:35:12 2022] POST /identity/v3/projects => generated 312 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:12.480708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 24/46] 10.222.0.72 () {60 vars in 1131 bytes} [Wed Aug 24 07:35:12 2022] POST /identity/v3/roles => generated 223 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:13.606895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd73-e4b2-4f43-8633-0297191953ad None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:13.607425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd73-e4b2-4f43-8633-0297191953ad None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:13.607425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd73-e4b2-4f43-8633-0297191953ad None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:13.608836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 23/47] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:13.614128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:13.614536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:13.614675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:13.632581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:13.633803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:13.698996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 25/48] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:13.963409 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadd73-e4b2-4f43-8633-0297191953ad None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.068763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadd73-e4b2-4f43-8633-0297191953ad None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.085743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:14.102781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.103412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.103692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.103963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.115944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:14.122692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:14.123537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:14.124290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 24/49] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:14.129869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.164858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:14.231406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892697b-5d1f-45fd-b6d7-68bade70d5e8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.264713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.265310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.265593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.265876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.266546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:14.266852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:14.268238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd06d-8926-45fd-a4fa-fdc6282bd8c5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:14.276859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 26/50] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:14.282452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.383672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40843-7f72-43d8-ab63-ed3bf6066793 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.397251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f310-af4b-4060-b83e-60e6efa3b0a5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.397541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f310-af4b-4060-b83e-60e6efa3b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.397825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7f310-af4b-4060-b83e-60e6efa3b0a5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.399162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 27/51] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:14.404230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.404469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.404781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.420379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.421048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.421332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.421607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.422775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:14.422963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:14.423965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:14.424420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:14.426067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 25/52] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:14 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:14.431461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.477944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:14.497342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 28/53] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:14.532387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6aabf-4ef5-40e1-95d4-0f35036619ad None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.565638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.566270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.566550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.566822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.567931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:14.569412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:14.570942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 26/54] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:14.576762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.603627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:14.681007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1791db-ed0c-4ab2-a125-fe0eacbae22d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.715049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:14.715653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:14.715930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:14.716200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:14.716957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.domain.id=default, user.id=691d549b6a8240cfaab77208fe5f61bd) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:14.717267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:14.718705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:14.724300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 29/55] 10.222.0.72 () {56 vars in 1322 bytes} [Wed Aug 24 07:35:14 2022] GET /identity/v3/role_assignments?user.id=691d549b6a8240cfaab77208fe5f61bd&scope.domain.id=default&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:14.770178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:14.977805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b9d5-77b4-486f-aa65-4cbd13658180 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:14.991870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:15.011240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:15.011867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:15.012147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:15.012417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:15.013542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:15.014989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7abf-e8b0-4bc9-a67a-14b74b4b39a4 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:15.016715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 27/56] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:15.025539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:15.114955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:15.124672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bee74-597e-417d-8fc0-5f6e95da3f2d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:15.156963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:15.157574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:15.157855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:15.158128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:15.158764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:15.160203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6478-9a2a-49d0-8df0-924e29678129 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:15.230996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 30/57] 10.222.0.72 () {60 vars in 1140 bytes} [Wed Aug 24 07:35:15 2022] POST /identity/v3/projects => generated 298 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:17.113885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab411d-3e56-496b-aa55-e61658305184 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:17.114424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab411d-3e56-496b-aa55-e61658305184 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:17.114424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab411d-3e56-496b-aa55-e61658305184 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:17.115825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 28/58] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:17.121419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:17.121837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:17.121963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:17.140912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:17.142172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:17.206255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 31/59] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:17.475361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab411d-3e56-496b-aa55-e61658305184 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:17.575944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab411d-3e56-496b-aa55-e61658305184 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:17.609149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:17.609773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:17.610057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:17.610332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:17.611440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:17.612923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99839179-26fe-4f49-929b-5a136b0e6cf3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:17.614484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 29/60] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:17.620056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:17.688237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:17.716257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1cad6-caf5-44c5-bfa0-7947b2bc6259 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:17.749492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:17.750098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:17.750375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:17.750647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:17.751297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:17.752758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6f30f-e9f9-4816-be4f-e192e39bf347 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:17.887018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 32/61] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:17 2022] POST /identity/v3/users => generated 275 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:21.213686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619d04-7b98-43e0-9b7f-a7582664086b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.214769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619d04-7b98-43e0-9b7f-a7582664086b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.215256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619d04-7b98-43e0-9b7f-a7582664086b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.216917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 30/62] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:21.222512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b5f5-29f2-4f7d-b03c-7d0157d80b62 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.223344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b5f5-29f2-4f7d-b03c-7d0157d80b62 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.223988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02b5f5-29f2-4f7d-b03c-7d0157d80b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.249121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ee02b5f5-29f2-4f7d-b03c-7d0157d80b62 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.250655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ee02b5f5-29f2-4f7d-b03c-7d0157d80b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:21.354081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 33/63] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:21.466834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d6c5-2254-4512-a027-e4ad737e4f3e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.467430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d6c5-2254-4512-a027-e4ad737e4f3e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.467906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553d6c5-2254-4512-a027-e4ad737e4f3e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.469591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 31/64] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:21.475367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b96ca-d297-44ff-9a1c-9e4b9837978d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.475896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b96ca-d297-44ff-9a1c-9e4b9837978d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.476367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b96ca-d297-44ff-9a1c-9e4b9837978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.487937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e21e-f6b4-4641-ab12-54b9d6d00723 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.488495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e21e-f6b4-4641-ab12-54b9d6d00723 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.489054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391e21e-f6b4-4641-ab12-54b9d6d00723 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.490593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 32/65] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:21.495512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229f1c-d0eb-462c-bfe1-c3c6008a7121 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.495793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229f1c-d0eb-462c-bfe1-c3c6008a7121 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.496070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229f1c-d0eb-462c-bfe1-c3c6008a7121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.517075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-694b96ca-d297-44ff-9a1c-9e4b9837978d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.517251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-64229f1c-d0eb-462c-bfe1-c3c6008a7121 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.518397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-64229f1c-d0eb-462c-bfe1-c3c6008a7121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:21.518653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-694b96ca-d297-44ff-9a1c-9e4b9837978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:21.580056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:21.581694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:21.596335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 34/66] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:21.598129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981b1b8-dcb4-4400-bed9-7bddf7fc367a None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.598540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981b1b8-dcb4-4400-bed9-7bddf7fc367a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.598940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981b1b8-dcb4-4400-bed9-7bddf7fc367a None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.600301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 35/67] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:21.606685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.607600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.608084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.616273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 33/68] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:21.617837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6b3e-83a7-4b07-b6e9-23d31996fd53 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.618349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6b3e-83a7-4b07-b6e9-23d31996fd53 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.618835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d6b3e-83a7-4b07-b6e9-23d31996fd53 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.620526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 34/69] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:21.625877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.626377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.626871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.641656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.642987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:21.646314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:21.647648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:21.719570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 35/70] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:21.726853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:21.806093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 36/71] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:21.840119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedc0ae-4904-4d3d-b50d-d8d2bb3006e9 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:21.851057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:21.876260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:21.876928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:21.877210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:21.877486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:21.887696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:21.892693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:21.893694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:21.894614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:21.896428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 36/72] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:21.902258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:21.905165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.037459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f7bd4-bad0-4365-990b-ae49d31957bb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.071098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aefd6e-385e-40f8-bed6-4f0d123924aa None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.081352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.081996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.082276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.082549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.092614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.093789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:22.100250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:22.106926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 37/73] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:22.107681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.110766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.111381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.111661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.111934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.112604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:22.112934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.114477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.124085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 37/74] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.124914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.139204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.208101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c966e6-8ed5-44f6-949d-c8d2ec6fc1d6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.224168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f6945-9814-414e-a359-44ff1ecc7e8d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.241259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.241889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.242172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.242444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.252980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:22.257679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:22.258511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:22.263268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:22.263843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.263843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.263989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 38/75] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:22.264052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.264052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.264180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.264245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.264245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.267728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 38/76] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.268569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.316417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.351553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.360842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f31148-9a42-4e99-aa97-c97b3d4caee7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.367278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55726e2c-1bd2-4d69-9504-55c8822a969a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.396362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.397035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.397317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.397588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.398254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:22.398567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.399980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.405918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.406562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.406847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.407125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.411663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 39/77] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.412490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.418681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:22.424121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:22.425204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:22.425990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 39/78] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:22.426606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.519939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57483b60-4745-48cf-ad01-5fbda2de90eb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.530084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38758-c354-46c3-a6cc-ec31b3116265 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.538832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.553339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.553974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.554279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.554553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.564984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.565840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.566286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.566869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.567500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:22.567813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.569234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.570207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.571669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.573465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 40/79] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.574237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.586466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 40/80] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.586466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.586690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.674188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61be58-9fff-40a2-a63a-c2736b5bd211 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.684238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a6786-b208-4bb3-b42f-8fc908fd8ac5 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.708170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.708808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.709121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.709397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.710047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.711523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.717811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.718499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.718785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.719067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.720230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.721740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.723337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 41/81] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.724164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.763990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.795484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 41/82] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:22 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:22.797616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.828544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f3c32-7133-4283-8b8e-1d0dabc0475e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.868674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.869346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.869629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.869901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:22.870584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:22.870888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:22.872370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:22.882254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:22.883425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 42/83] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:22.885193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:22.925506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb3134-82ac-47a8-9094-58598ae79e6f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:22.966421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:22.967044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:22.967329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:22.967601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.000144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.003451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78babebe-84ba-4f56-abd4-cba8f9f7317e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.049546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.050194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.051105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.051105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.051875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.053465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.056182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 43/84] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:23.057062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.064104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.066047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.142553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.176063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3300bd-2178-4a00-bced-1f36534250bc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.181113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 42/85] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:23.181818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.230010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.230701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.230983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.236676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.283012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.298551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229067-7be2-4d00-8bde-7b4aa1144b52 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.337205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.337555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.338515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.338855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.339128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.339594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.340353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.341918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.344009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 43/86] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:23.344962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.384114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.465916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62a6ba-73a4-464c-9f28-85e4209fe49c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.471950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 44/87] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:23.473148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.510151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.510757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.511040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.511316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.549661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.583922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202929c6-67c2-43df-93fd-b137b7ff94ca None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.606570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.608066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.628236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.628937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.629218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.629490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.631154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.632936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.635464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 45/88] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:23.636144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.655945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.753048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 44/89] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:23.754299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.757291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea9464-c326-4d1b-a925-85764099353b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.803934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.804562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.804933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.805241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.810967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:23.873160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b86e1-05c7-426d-90b9-837e91a44b79 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:23.895408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.897281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.925896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:23.926539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:23.926819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:23.927092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] PATH_INFO: `/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:23.928853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] RBAC: Authorizing `identity:get_project(project_id=7c8a84d8b7a640b1bdd0319744a48ac7)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:23.930301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:23.937713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 45/90] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7 => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:23.937713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:23.943803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:24.039326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 46/91] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:24.040068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:24.071482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138526d-c016-4651-ab0f-484e0654d30e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.089232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:24.110332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.111258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.111537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.111868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.112609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:24.113007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.114915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.127128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 46/92] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:23 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=0bb62a6e624c40b4832f54f88761160b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:24.128025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:24.207521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07361758-29ef-4b26-8679-bb48a027108a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.270793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.271546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.271730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.272009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.274056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.276153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.276586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:24.279471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 47/93] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:24 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:24.280374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:24.326430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cab56-090a-4004-9c73-7c67b238c6ce None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.402283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e735a112-da4e-4c4c-9d00-56bbae5fd298 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.404770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:24.413371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.414006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.414294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.414576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.415324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=7c8a84d8b7a640b1bdd0319744a48ac7, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:24.415763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.421459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.433966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 47/94] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:24 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=7c8a84d8b7a640b1bdd0319744a48ac7&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:24.434903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:24.443486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.444152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.444434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.451247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.452623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.454159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.459218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 48/95] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:24 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:24.465491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:24.659449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f2c0c-7648-4c0e-ad44-2a81df47964e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.679588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b188-235e-4583-904a-24ae7b6ed0d0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:24.717372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:24.724862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.725491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.725774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.726050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.726834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:24.729136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.729300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:24.729893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:24.730176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:24.730457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:24.731204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=691d549b6a8240cfaab77208fe5f61bd) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:24.731307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce0adc-89f4-4645-9df7-eb020932bcb2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.731519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:24.732989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635eec69-0765-4b75-8aed-fc8c12e7081b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:24.737447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 48/96] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:24 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:24.745942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 49/97] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:24 2022] GET /identity/v3/role_assignments?user.id=691d549b6a8240cfaab77208fe5f61bd&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:25.039335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be671-c699-4edd-9ab9-b35047bc0012 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:25.039664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be671-c699-4edd-9ab9-b35047bc0012 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:25.039832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055be671-c699-4edd-9ab9-b35047bc0012 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:25.041540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 49/98] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:25.047324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff3ec-5060-4e05-b49c-b8f81eba2922 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:25.047324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff3ec-5060-4e05-b49c-b8f81eba2922 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:25.047572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff3ec-5060-4e05-b49c-b8f81eba2922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:25.070797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-acdff3ec-5060-4e05-b49c-b8f81eba2922 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:25.072074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-acdff3ec-5060-4e05-b49c-b8f81eba2922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:25.106734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:25.157077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 50/99] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:25.206778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783a6c1-e1f0-47e2-88d3-70225b429c56 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:25.207088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783a6c1-e1f0-47e2-88d3-70225b429c56 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:25.207339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783a6c1-e1f0-47e2-88d3-70225b429c56 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:25.208706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 50/100] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:25.220142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:25.220431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:25.220737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:25.242995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:25.244767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:25.331564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 51/101] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:25.645597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783a6c1-e1f0-47e2-88d3-70225b429c56 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:25.764640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783a6c1-e1f0-47e2-88d3-70225b429c56 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:25.817414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:25.818072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:25.818353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:25.818625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:25.829914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:25.835100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:25.835920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:25.836688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 51/102] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:25.842692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:25.867913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:25.972189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b646cee-359d-41b0-a0b3-b04684ccf7e4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.006938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.018570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.019198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.019674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.019830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.020410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:26.020736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.022337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.040169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 52/103] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:26.045758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:26.061580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.225608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea6af-de78-48d7-9e42-5bdad1da6cff None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.231003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afa66f-ab54-4fe8-981e-64c8d2a78d73 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.322103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.322780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.323050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.323329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.327533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.329067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.329457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.330130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.330419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.330697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] PATH_INFO: `/v3/users/182e402dbf87459c90b0a50d44abcf15` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.331109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 52/104] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:26.336685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:26.359113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] RBAC: Authorizing `identity:get_user(user_id=182e402dbf87459c90b0a50d44abcf15)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.361263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.363193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 53/105] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:26 2022] GET /identity/v3/users/182e402dbf87459c90b0a50d44abcf15 => generated 289 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:26.368766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:26.406473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.498927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65448d09-29b7-416c-960d-0d91ef103dcd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.537023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.537952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.538502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.538921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.606838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf60f90-f853-45ca-bc9f-434e99e11ec4 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.611552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.613260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc992dd-9ecb-488c-8694-440675fa8a0b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.656802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.657856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.685972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.686934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.687820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.688304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.689256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.690946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc4b8d-59ee-49af-a64f-42cef796db6f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.761535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 54/106] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:26.763283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e4790-3692-4559-8191-424affedd9cb None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.763787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e4790-3692-4559-8191-424affedd9cb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.764274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e4790-3692-4559-8191-424affedd9cb None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.765775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 55/107] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:26.767226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e4790-3692-4559-8191-424affedd9cb None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:26.796410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 53/108] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:35:26 2022] POST /identity/v3/projects => generated 302 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:35:26.797997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.798464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.798943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.828912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:26.830413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:26.934946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e4790-3692-4559-8191-424affedd9cb None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:26.960043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:26.989659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:26.990314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:26.990597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:26.990872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:26.992201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:26.993732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:26.995390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 56/109] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:26 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:27.003432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:27.042322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 54/110] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:27.134768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76e4b-633a-4e5e-939f-85969d1af56a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:27.174768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:27.175394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.175674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.176007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.176779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=182e402dbf87459c90b0a50d44abcf15) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:27.177136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:27.178595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c7d8e-62a9-4521-8b2e-c27808e63ed2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:27.185006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 57/111] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:27 2022] GET /identity/v3/role_assignments?user.id=182e402dbf87459c90b0a50d44abcf15&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=273e4fba5ef141888990da15821d9dfb => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:27.432620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:27.518166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f9f0-a889-4591-99ae-702ead35fa2d None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.518667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f9f0-a889-4591-99ae-702ead35fa2d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.518739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f9f0-a889-4591-99ae-702ead35fa2d None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.520212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 58/112] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:27.526040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4907e3-eab9-45b5-9940-64c10715c83e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.526321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4907e3-eab9-45b5-9940-64c10715c83e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.526595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4907e3-eab9-45b5-9940-64c10715c83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.553204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7a4907e3-eab9-45b5-9940-64c10715c83e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:27.554430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7a4907e3-eab9-45b5-9940-64c10715c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:27.602717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506c336-3854-4f57-bc2b-656ddd440ee1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:27.605884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:27.625102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:27.650148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 59/113] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:27.653811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa866-a551-489c-a138-5a07f7aa0243 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.654926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa866-a551-489c-a138-5a07f7aa0243 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.655152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa866-a551-489c-a138-5a07f7aa0243 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.656881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 60/114] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:27.662039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee50d9-afcd-47e5-80c3-a8cbf5ffba6c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.662317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee50d9-afcd-47e5-80c3-a8cbf5ffba6c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.662591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee50d9-afcd-47e5-80c3-a8cbf5ffba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.690732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:27.691407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.691690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.691938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.693272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f9ee50d9-afcd-47e5-80c3-a8cbf5ffba6c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:27.694466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f9ee50d9-afcd-47e5-80c3-a8cbf5ffba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:27.714767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:27.719606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:27.720448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:27.721284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 55/115] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:27.726864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:27.824446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 61/116] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:27.825752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23dc95-a6c7-4c33-86eb-23712aebaf23 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.826038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23dc95-a6c7-4c33-86eb-23712aebaf23 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.826314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23dc95-a6c7-4c33-86eb-23712aebaf23 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.828485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 62/117] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:27.835605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.835885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.836161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.862959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:27.864160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:27.934719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 63/118] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:27.942668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a341-3c1a-4327-a5df-641d7a31cb5e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:27.994395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:27.995017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:27.995300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:27.995592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:27.996243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:27.996552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:27.998026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.011949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 56/119] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:28.017631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.063689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.072411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.120759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.185272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573cad4-f725-4348-8275-994ec6ab4781 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.243741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.244393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.244691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.244974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.248965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:28.250435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.252064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 64/120] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:28.252859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.272090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66156ae4-a859-4588-948b-6c239aa426ed None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.334254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.334905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.335192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.335474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.358724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:28.363956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:28.364859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:28.365589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 57/121] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:28.366202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.394749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d360c-a142-4599-832f-17ac77b02a1e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.435566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.436641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.437172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.437718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.440854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.449736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:28.455112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:28.456044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 65/122] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:28.464779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.465516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.483422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b323-1627-4c5f-814b-6a13bd96a305 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.520543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.521839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.522249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.522712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.524006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:28.526151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.528275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 58/123] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:28.533576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.572380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa040a7-e473-48a7-919b-5c07f1b45a70 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.610798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.611699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.612376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.612825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.626163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:28.633209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:28.634795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:28.635739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:28.638150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 66/124] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:28.639530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.648781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b28b8-30a4-4513-b817-7642140e65b2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.675363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.698561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.699828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.700289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.700911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.702128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:28.702580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:28.704119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.717969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 59/125] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:28.719175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.741480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.770973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50afe6-5286-4a42-bd7f-16619daa60c4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.820131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.822262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.822723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.823140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.824215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:28.825103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:28.827181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.830994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f39cf-49e7-4d8c-9d11-656a06f19ac7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.838932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 67/126] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:28.839971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:28.872862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:28.874027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:28.874413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:28.874796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/c6a2704c79154a9ab284fa13a4e9e446/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:28.878106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, user_id=c6a2704c79154a9ab284fa13a4e9e446, role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:28.880689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:28.941286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:28.953231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 60/127] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:28 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/c6a2704c79154a9ab284fa13a4e9e446/roles/0bb62a6e624c40b4832f54f88761160b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:28.956965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737b1e-31e2-4946-8c3f-9e1603920031 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:28.958229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.029582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.030552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.031230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.032567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.033745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:29.035574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.038158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.056780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 68/128] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.059113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.125021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.156368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2137d-6c7b-414c-a871-55a7ac4a3f08 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.198431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.199428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.199927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.200381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.202186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.204547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.207045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 61/129] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:28 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.208222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.224728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d47e46d-e3bc-42bf-ab2b-5b32f541b434 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.263711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.264597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.264978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.265417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.266612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.268602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.270223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 69/130] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:29 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.273705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.298452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.338562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3847-35cd-4208-a425-c56b5751df36 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.375947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.376904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.377012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.377411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.378687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.380294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.382043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 62/131] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:29 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.382955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.398330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.416677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06434ae1-bdab-4ada-8bc5-604d33452a43 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.455142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.455829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.456177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.456520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] PATH_INFO: `/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.457768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=7c8a84d8b7a640b1bdd0319744a48ac7)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.459359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39a700-41f4-49fb-98b1-eacd1647c9c7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.461471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 70/132] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:29 2022] GET /identity/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.462750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98208-d0c7-4c29-a688-07b1c3573406 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.463098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98208-d0c7-4c29-a688-07b1c3573406 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.463449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b98208-d0c7-4c29-a688-07b1c3573406 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.464933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 71/133] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:29.465581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b98208-d0c7-4c29-a688-07b1c3573406 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.509702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38caa3f-05ae-4bb5-8a38-0f6df6245c83 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.560343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.564026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.564698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.565268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.565550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.566874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.568961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbeb2f-6901-4bf7-a30c-555c31d3eb01 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.571765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 63/134] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:29 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.573051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.573334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.573609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.581834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b98208-d0c7-4c29-a688-07b1c3573406 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.596283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:29.597532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:29.630099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.631870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.632353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.632851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.634203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.636914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.643009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 72/135] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:29 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:29.643009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.673456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 64/136] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:29.674191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.690482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.767955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236dc20-a2cb-4506-9259-1aa88986c866 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.792374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22cc5a-9cf8-414e-a1b6-9d85ba1e2f3b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:29.807488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.808274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.808561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.808863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] PATH_INFO: `/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7/users/c6a2704c79154a9ab284fa13a4e9e446/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.810983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c8a84d8b7a640b1bdd0319744a48ac7, user_id=c6a2704c79154a9ab284fa13a4e9e446, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.812560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f960ff5-506c-42a4-9545-b6769f1b2790 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.829638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:29.830304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.830589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.830866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/c6a2704c79154a9ab284fa13a4e9e446/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.832911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, user_id=c6a2704c79154a9ab284fa13a4e9e446, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:29.834428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac07934-9d37-4c07-b65c-44fe407e1afe None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:29.914789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 73/137] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:29 2022] PUT /identity/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7/users/c6a2704c79154a9ab284fa13a4e9e446/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:29.916334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769fc6-47bd-479b-ab02-575d4b2bab4c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.916622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769fc6-47bd-479b-ab02-575d4b2bab4c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.916924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f769fc6-47bd-479b-ab02-575d4b2bab4c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:29.918311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 74/138] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:29.922780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f769fc6-47bd-479b-ab02-575d4b2bab4c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:29.948189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:29.984592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 65/139] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:29 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/c6a2704c79154a9ab284fa13a4e9e446/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:29.992519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:29.993134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:29.993956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.044461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:30.048257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:30.112886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f769fc6-47bd-479b-ab02-575d4b2bab4c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:30.151476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:30.155591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:30.156077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 66/140] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:30.156767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:30.157183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:30.157599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/691d549b6a8240cfaab77208fe5f61bd/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.157932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:30.166831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, user_id=691d549b6a8240cfaab77208fe5f61bd, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:30.168574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:30.273989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 75/141] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:29 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/691d549b6a8240cfaab77208fe5f61bd/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:30.277756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2c2ef-c3cb-4156-bdfa-f35c08cdc2d3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:30.320608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:30.321585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:30.322003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:30.322372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.323675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:30.325311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:30.327253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 67/142] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:30.333197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:30.421277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:30.448886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aac3f6-c559-4255-9571-c41b587840e9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:30.496908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:30.497545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:30.497826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:30.498147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.498830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:30.500369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:30.601860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:30.630524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 76/143] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:30 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:30.690427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:30.727339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52998100-d3d9-4727-85fc-e8c66facd5ec None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:30.776044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:30.776837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:30.777121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:30.777394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.793393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:30.798599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:30.805124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:30.806486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 68/144] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:30.809384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:30.947348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739c63-9fbc-41b5-8b2d-c3fb8c9b1ba9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:30.985168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:30.991004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:30.991624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:30.991905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:30.992175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:30.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:30.993218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:30.994644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:31.008360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 77/145] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:31.014761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:31.162774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba303b9-33a0-481b-a163-e7c9e31dc00b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:31.194969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:31.205147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:31.205725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:31.206010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:31.206351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] PATH_INFO: `/v3/users/182e402dbf87459c90b0a50d44abcf15` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:31.207546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] RBAC: Authorizing `identity:get_user(user_id=182e402dbf87459c90b0a50d44abcf15)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:31.209023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:31.217072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 69/146] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:31 2022] GET /identity/v3/users/182e402dbf87459c90b0a50d44abcf15 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:31.218077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:31.359202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc07f5-9803-41b5-988e-bc51aca6808a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:31.418262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:31.418910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:31.419193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:31.419464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:31.420622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:31.422112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cf8d7-a520-4ee3-8427-5c8d6c2556b6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:31.424011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 78/147] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:31 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:31.429963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:31.560720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8e10f-1d01-488f-b177-73a739ad2b2a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:31.599666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:31.600794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:31.601279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:31.601746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/182e402dbf87459c90b0a50d44abcf15/roles/273e4fba5ef141888990da15821d9dfb` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:31.604434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, user_id=182e402dbf87459c90b0a50d44abcf15, role_id=273e4fba5ef141888990da15821d9dfb)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:31.606145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5f8d-4cdf-4de7-bb60-a75050909c62 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:31.681291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 70/148] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:31 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/users/182e402dbf87459c90b0a50d44abcf15/roles/273e4fba5ef141888990da15821d9dfb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:31.734732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6044d6-04b9-4669-8a18-75a526bda268 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:31.735237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6044d6-04b9-4669-8a18-75a526bda268 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:31.735442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6044d6-04b9-4669-8a18-75a526bda268 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:31.737210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 79/149] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:31.742625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55554d2a-a72b-47ab-9bad-05b5c9e5ef78 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:31.743243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55554d2a-a72b-47ab-9bad-05b5c9e5ef78 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:31.743712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55554d2a-a72b-47ab-9bad-05b5c9e5ef78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:31.766276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-55554d2a-a72b-47ab-9bad-05b5c9e5ef78 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:31.767825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-55554d2a-a72b-47ab-9bad-05b5c9e5ef78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:31.842168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:31.860576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 71/150] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:32.285705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876174-73b2-431a-9628-98b8fc6e2a42 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:32.286038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876174-73b2-431a-9628-98b8fc6e2a42 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:32.286206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e876174-73b2-431a-9628-98b8fc6e2a42 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:32.287536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 80/151] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:32.299299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:32.299895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:32.300366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:32.323064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:32.324696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:32.382321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876174-73b2-431a-9628-98b8fc6e2a42 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:32.424483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 72/152] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:32.481778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:32.533348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e876174-73b2-431a-9628-98b8fc6e2a42 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:32.572460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:32.573122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:32.573399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:32.573674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:32.586429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:32.591590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:32.592414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:32.593198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 81/153] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:32.598642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:32.713074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe8d7a-9610-421d-bdf1-8c488d7d34fb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:32.730879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:32.748964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:32.758261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:32.762514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 73/154] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:32.768111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:32.887640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd5b5f-1c2f-4469-9790-80dee16cb96f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:32.912167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:32.929468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:32.930112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:32.930390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:32.930726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:32.931898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:32.933637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:32.935235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 82/155] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:32 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:32.941002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.044290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.080894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c799e693-63d6-4541-b55d-d8d57bc52d6e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:33.140516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:33.141141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.141423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.141696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.223959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:33.227907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87cb33-6793-4546-b82e-7e90c79ed850 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:33.277036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:33.277347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:33.278034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.278317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.278518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b9b0e-f69d-4cfd-88a4-31d4a73c2048 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:33.278637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.306358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-46501b22-779f-4f22-8513-741a66ba636a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:33.315999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:33.316893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:33.317705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 83/156] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:33.318787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb24a7-1e01-4063-8b3e-9f6b6ff55c61 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.318876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb24a7-1e01-4063-8b3e-9f6b6ff55c61 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.319154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb24a7-1e01-4063-8b3e-9f6b6ff55c61 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.320411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 84/157] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:33.324275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb24a7-1e01-4063-8b3e-9f6b6ff55c61 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.415498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:33.497031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 74/158] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:33.498302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.498584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.498858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.515470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb24a7-1e01-4063-8b3e-9f6b6ff55c61 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:33.540494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:33.541834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:33.567421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:33.568030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.568310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.568584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.569578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:33.569885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:33.572019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:33.583195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 85/159] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:33.584194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.698991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 75/160] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:33.700058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.723129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:33.760803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f6bcd-b215-4c21-a861-4eb763a119fe None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:33.781032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:33.836164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:33.837166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.837577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.838061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.845703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:33.847960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:33.850562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 86/161] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:33.864498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:33.904294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a9464-59fd-4568-bdac-099db4c2e0a6 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:33.973371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:33.973969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:33.974206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:33.974484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:33.976567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:33.978594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:33.980360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 76/162] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:33.986048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:34.120327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50760307-000c-4501-92da-4b6db9c6669c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:34.122782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:34.176419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4dfd25-734a-4097-a85c-6569ee27ac0c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:34.214446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:34.215360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.215785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.216202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.217395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:34.217872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:34.220186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf4a4-b493-477d-b4a1-f61c09abee05 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:34.240178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 87/163] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=0bb62a6e624c40b4832f54f88761160b => generated 608 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:34.241543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f33e15-c6c2-487a-aedc-d74a5f054031 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.241975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f33e15-c6c2-487a-aedc-d74a5f054031 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.242384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f33e15-c6c2-487a-aedc-d74a5f054031 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.244500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 88/164] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:34.245587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f33e15-c6c2-487a-aedc-d74a5f054031 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:34.253966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:34.254611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.254895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.255191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.370492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:34.449962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:34.451470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:34.480622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f33e15-c6c2-487a-aedc-d74a5f054031 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:34.529123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:34.530870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.531610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.532077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.552380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-29ff2d8a-b27e-47ea-bdfa-f960192268e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:34.559949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:34.560921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:34.561815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:34.563723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:34.565899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 89/165] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:34.567971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.568445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.569077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.593328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:34.595515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:34.598066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 77/166] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:34.598810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:34.702665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 90/167] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:34.703846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:34.718675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2ace8-896a-420c-a106-636cef87e52b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:34.764282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:34.764937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.765223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.765494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:34.766480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:34.766782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:34.767868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:34.777617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 78/168] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:34.785784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:34.859983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eff248-f200-4351-bfbd-797de41a3033 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:34.867197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:34.910887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:34.911817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:34.912263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:34.912755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] PATH_INFO: `/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:34.914357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] RBAC: Authorizing `identity:get_project(project_id=7c8a84d8b7a640b1bdd0319744a48ac7)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:34.916447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:34.918287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 91/169] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:34 2022] GET /identity/v3/projects/7c8a84d8b7a640b1bdd0319744a48ac7 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:34.924714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:35.026770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ff20-3606-4966-9332-f4983c4bf36c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.082833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:35.091555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930616f-6894-4769-8998-f12b09de75fe None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.125386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.126040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.126326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.126603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] PATH_INFO: `/v3/users/c6a2704c79154a9ab284fa13a4e9e446` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.127801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] RBAC: Authorizing `identity:get_user(user_id=c6a2704c79154a9ab284fa13a4e9e446)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:35.132189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:35.132189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 79/170] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:34 2022] GET /identity/v3/users/c6a2704c79154a9ab284fa13a4e9e446 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:35.136965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:35.142111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.143294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.143767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.149206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.150622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=7c8a84d8b7a640b1bdd0319744a48ac7, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:35.151125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:35.152872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:35.161327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 92/171] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:34 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=7c8a84d8b7a640b1bdd0319744a48ac7&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:35.302373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf604c0-454b-4dd1-aaa4-1369ffb96d58 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.313112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:35.364002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.364693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.364984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.365262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.406588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:35.508867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:35.510701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85108aaa-ba55-4a12-b7aa-45e6f9264523 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:35.582279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238f07-f243-4fc9-a4b1-ffbba944c026 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.641477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.642057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.642352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.642783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.654348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 80/172] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:35.656248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3e62-80fb-437d-8657-d979abfeb753 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.656767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3e62-80fb-437d-8657-d979abfeb753 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.657257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd3e62-80fb-437d-8657-d979abfeb753 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.658734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 81/173] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:35.661705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:35.665364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.666129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.666776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:35.672638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:35.673572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:35.674347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 93/174] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:35.674977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:35.685872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:35.693555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:35.695649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:35.718111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:35.783390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 82/175] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:35.784472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:35.842784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d05f7-8193-443c-ba30-8628d9b52771 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.896287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.897332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd188-e758-41ee-abdf-e0cb6b4e7c6b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:35.897989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.898479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.898993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.906405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:35.909318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf2763-9405-44c4-8f4e-ff5e09dd7ab6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:35.911506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 94/176] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:35 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:35.913064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8815-4090-4718-9938-98bda12744d4 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.913588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8815-4090-4718-9938-98bda12744d4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.914070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c8815-4090-4718-9938-98bda12744d4 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.916834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 95/177] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:35.922843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.923450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.924023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.958423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:35.960635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:35.975607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:35.980714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:35.987993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:35.996524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 83/178] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:36.002096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.109444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 96/179] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:36.109444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.149197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.216327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d81c65-2e30-4dae-87df-a0dd2a4a8cec None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.258356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.259010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.259289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.259560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.260230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=c6a2704c79154a9ab284fa13a4e9e446) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:36.260542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:36.262087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef1f0b-17db-41e8-b365-4b653554f992 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:36.263039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b57ba-827c-44d6-83f4-ad3fa73f1927 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.269700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 84/180] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:35 2022] GET /identity/v3/role_assignments?user.id=c6a2704c79154a9ab284fa13a4e9e446&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:36.271023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0fc27-8d35-4c95-b3b1-6d3549a16168 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.271389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0fc27-8d35-4c95-b3b1-6d3549a16168 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.271633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0fc27-8d35-4c95-b3b1-6d3549a16168 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.273125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 85/181] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:36.275472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ae6f-cb77-4278-846f-8a4c3db1b6ab None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.275751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ae6f-cb77-4278-846f-8a4c3db1b6ab None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.276023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37ae6f-cb77-4278-846f-8a4c3db1b6ab None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.277697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 86/182] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:36.278395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37ae6f-cb77-4278-846f-8a4c3db1b6ab None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.316388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.317072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.317351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.317626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] PATH_INFO: `/v3/users/691d549b6a8240cfaab77208fe5f61bd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.318826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] RBAC: Authorizing `identity:get_user(user_id=691d549b6a8240cfaab77208fe5f61bd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:36.320327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5edf4-92ab-432b-8500-2e656e37ca43 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:36.327179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 97/183] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/users/691d549b6a8240cfaab77208fe5f61bd => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:36.328803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557970ff-8676-464d-bf1e-565162107709 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.329086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557970ff-8676-464d-bf1e-565162107709 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.329425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557970ff-8676-464d-bf1e-565162107709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.354750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-557970ff-8676-464d-bf1e-565162107709 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:36.356954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-557970ff-8676-464d-bf1e-565162107709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:36.364576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.416535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37ae6f-cb77-4278-846f-8a4c3db1b6ab None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.456881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 98/184] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:36.458251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.458382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.458566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.458813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.458919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.459165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.459438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.475377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:36.478240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:36.479433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:36.481462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:36.482357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:36.483197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 87/185] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:36.483865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.500348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.567834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 99/186] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:36.569178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a42e-3e94-4402-a824-3945113fac6c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.623131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.637288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.638293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.639035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.639753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.692833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22affc9c-32a1-46e8-8a71-65e4b0e3f0e9 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.728001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:36.729216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.729615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:36.729851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.730121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.730383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.741641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:36.746441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:36.747291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:36.748171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 100/187] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:36.749307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.762154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.843410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 88/188] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:36.844089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.851749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1566df-37b3-4dcb-8585-ccb19a865066 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.868353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.885727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.886340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.886618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.886891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.887557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:36.887864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:36.889298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:36.897712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 101/189] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:36.898342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:36.945577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14e37f-fa28-4ba5-a397-e28d11a04c68 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:36.967767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:36.979311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:36.979932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:36.980207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:36.980470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:36.990353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:36.995010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:36.995886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 89/190] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:37.001730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.002625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1b0bb-8867-4cfd-8ee6-4202149cd8cd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.031834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.032445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.032761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.033040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.033708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:37.034010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.035388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.043295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 102/191] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.044078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.085761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.103091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ec2e7-dfbd-417c-aec9-0291d99fe6c7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.137346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.137963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.138233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.138503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.139611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.141082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.142690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 90/192] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:36 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.143441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.148933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55274f-a679-49e5-8226-17c41c7ad91f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.183480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.184355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.184634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.184938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.186054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.187510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.189109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 103/193] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.189109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.189109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.243148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115f9d-0b2b-4a66-aaf6-eff8d04ca52a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.278637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.280558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.280865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.281132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.289969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6dfd5c-41a1-4611-8ca0-0b89fbbd9980 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.299853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.301347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.303172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 91/194] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.303785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.314978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.328833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.329447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.329801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.330075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.330746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:37.331053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.332429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.340389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 104/195] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.341246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.404103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687fd6-3e60-4f53-9013-f805534d2b8b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.404315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.439509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.440134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.440413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.440707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] PATH_INFO: `/v3/users/182e402dbf87459c90b0a50d44abcf15` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.441818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4073f67-5c0e-4ee9-a9a2-49fa5eb6206c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.441942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] RBAC: Authorizing `identity:get_user(user_id=182e402dbf87459c90b0a50d44abcf15)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.443282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.444894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 92/196] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/users/182e402dbf87459c90b0a50d44abcf15 => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.448930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.485068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.485677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.485959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.486233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.486966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=691d549b6a8240cfaab77208fe5f61bd) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:37.487274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.488788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.495067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 105/197] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/role_assignments?user.id=691d549b6a8240cfaab77208fe5f61bd&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.495780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.532099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.559915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b7a79-b081-46d6-885e-e0b44adfa3eb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.598316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.598955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.599294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.599632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.600339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.601844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.617154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb1fa-37b8-4d8f-876c-91f44b6d1677 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.645278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.652822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.653439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.653711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.653977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.698115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 93/198] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:37 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:37.699239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.729344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.730833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.837721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3943f-0748-4c01-b013-fdcd7711dc44 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:37.858494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 106/199] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:37.859732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.874310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.879394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:37.880603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:37.880603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:37.880603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:37.881677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:37.883130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:37.885051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 94/200] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:37.885866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:37.896561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:37.975729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0deada-25ca-43fe-931c-7c03c817a732 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.002119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0bf57-0324-4565-90ca-14bd8167315f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.028808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.029425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.029726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.030090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.043049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.043695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.043973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.044244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.046980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.048371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.050057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 95/201] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:38.050816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.111499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.113284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.136897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.143176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.174372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4929f00-59fb-449f-ac97-d5925d04cced None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.211187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.212470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.212778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.213049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.222770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 107/202] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:38.223852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.291245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.292805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.337277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303376d2-ff32-4a57-8b29-0d3dbf6776a1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.373128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.374590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.375416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.375766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.376085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.376197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.380673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=1e680d0f82ac4865a20de5ea136f2dee) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:38.380673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.380673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.385627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 108/203] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/role_assignments?user.id=1e680d0f82ac4865a20de5ea136f2dee&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:38.387563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.448509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 96/204] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:38.456118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.544089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7413-a920-4bd5-ae6a-2778da581361 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.606695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.607498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.608222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.608593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.610460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.611982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.614380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 109/205] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:38.620909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.644612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b7722-f717-40d9-b53d-35b89bcc3336 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.717134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.718051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.718333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.718602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.722388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.723858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.725724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 97/206] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:38.732010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:38.745389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.775089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:38.844555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f632c02-7a4f-4997-8bbb-37e56e8c41f7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.862835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56587e61-7261-45f8-ab8a-e3f92e4ac97f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:38.884116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.884754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.885034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.888672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.888672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=1e680d0f82ac4865a20de5ea136f2dee) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:38.888672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.888672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89addabb-b940-4992-8113-e0b027c2a022 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.894549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 98/207] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/role_assignments?user.id=1e680d0f82ac4865a20de5ea136f2dee&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=0bb62a6e624c40b4832f54f88761160b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:38.919587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:38.920875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:38.921664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:38.922346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:38.923371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3, user.id=182e402dbf87459c90b0a50d44abcf15) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:38.923999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:38.926203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7beaf7-aa8e-426a-b3a8-5a2d97a49bb6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:38.937137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 110/208] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:38 2022] GET /identity/v3/role_assignments?user.id=182e402dbf87459c90b0a50d44abcf15&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=273e4fba5ef141888990da15821d9dfb => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:39.872359 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f133d-a652-4fd9-a263-8aabc96e1f13 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:39.873406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f133d-a652-4fd9-a263-8aabc96e1f13 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:39.873886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f133d-a652-4fd9-a263-8aabc96e1f13 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:39.875714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 99/209] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:39.881400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:39.882169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:39.882640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:39.902222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:39.903759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:39.951243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908eb20a-7312-4232-aa0d-39c998f92329 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:39.951551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908eb20a-7312-4232-aa0d-39c998f92329 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:39.951784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908eb20a-7312-4232-aa0d-39c998f92329 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:39.953302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 100/210] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:39.958453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:39.958733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:39.959005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:39.980765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 111/211] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:39.981649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:39.982832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:40.050581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 101/212] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:40.249694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.307777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:40.327849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.356548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427b650-bf05-49c1-92c7-323d1b84e3dc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.357549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:40.394243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.394985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.395265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.395542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.406636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:40.411868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:40.412783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:40.413567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 112/213] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:40.420944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.440804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65a2fd-1aa3-4d26-ad0c-fd7cd711a0b4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.484776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.485408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.485684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.485950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.487058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:40.488515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:40.490193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 102/214] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:40.496068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.536855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1010d-6a8c-4cc4-9f76-a5823874d0fc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.557327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:40.579363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.579979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.580257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.580528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.581233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:40.581558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:40.582953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ba99f-d27e-4030-96d6-d526a6461e09 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:40.596034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 113/215] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:40.596034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499199cb-2286-4a1c-8061-e2abca021d3c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.596034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499199cb-2286-4a1c-8061-e2abca021d3c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.596034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499199cb-2286-4a1c-8061-e2abca021d3c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.596034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 114/216] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:40.597747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499199cb-2286-4a1c-8061-e2abca021d3c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.605037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:40.608904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b1e9-49db-47ed-b153-2c593c14d8f8 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.644696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.645314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.645587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.645851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.646503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:40.647933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:40.704619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499199cb-2286-4a1c-8061-e2abca021d3c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.737618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.738251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.738528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.738799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] PATH_INFO: `/v3/users/e6e7b3c03b534fd6813192545eaa442b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.751928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] RBAC: Authorizing `identity:get_user(user_id=e6e7b3c03b534fd6813192545eaa442b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:40.753429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a72e8-d9c8-4571-a71a-222222d47765 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:40.755070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 115/217] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:40 2022] GET /identity/v3/users/e6e7b3c03b534fd6813192545eaa442b => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:40.760201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823749-4c20-4991-bad4-f31d069909c9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.760201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823749-4c20-4991-bad4-f31d069909c9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.760201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01823749-4c20-4991-bad4-f31d069909c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:40.774732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-01823749-4c20-4991-bad4-f31d069909c9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:40.775988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-01823749-4c20-4991-bad4-f31d069909c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:40.800938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 103/218] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:40 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:35:40.807236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:40.810551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:40.863510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 116/219] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:40.934704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248788c-843f-43bf-b4a1-e05c4bc1787a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:40.973469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:40.974570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:40.975048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:40.975517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.002589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.009697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6ccf-e627-4068-85be-e6b961bc7ade None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.010556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6ccf-e627-4068-85be-e6b961bc7ade None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.017743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6ccf-e627-4068-85be-e6b961bc7ade None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.017743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 117/220] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:41.020265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.020791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.021278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.053577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:41.055139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:41.061492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.062986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.146598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 118/221] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:41.162529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.194887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 104/222] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:41.200295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.262472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.289537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.300730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86432f8-1244-4d85-bb5f-56384d9d523b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.324583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc039350-c392-42c7-a2e0-11fc59e4faff None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.353256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.353909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.354187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.354457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.368723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.369718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.370188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.370669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.371989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.372722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:41.374025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.376724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 105/223] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:41.383998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:41.384911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:41.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:41.387738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.389995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 119/224] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:41.402282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.505130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e4923-cff3-4748-b437-8ca7880955cc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.532837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.545332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.545947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.546222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.546492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.549519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=e6e7b3c03b534fd6813192545eaa442b) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:41.549519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.549519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.555286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 106/225] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/role_assignments?user.id=e6e7b3c03b534fd6813192545eaa442b&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:41.556099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.574949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e0bc-d1eb-4daa-96ad-c6ad61f8df34 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.622579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.623461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.624186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.624467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.625465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:41.625769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.627194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.640170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 120/226] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:41.647648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.668794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.685575 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da12c7-d563-4ebb-8d17-d8160051bb81 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.729225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.729871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.730218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.730474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.743109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:41.748581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:41.749687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:41.750626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 107/227] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:41.755703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.830026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f435-e3c2-4b4b-b252-a0268ac179c7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.845660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:41.875626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.877161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.877645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.881338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.883616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.885534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 121/228] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:41.893285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:41.912676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8bc4-d9c4-40c1-88f3-b9168c863ed0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:41.951527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:41.952536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:41.952850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:41.953125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:41.953804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:41.954107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:41.961117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:41.979185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 108/229] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:41.989635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:42.018805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:42.036692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b1a94-e547-404b-a8bf-0c1b42ccb2c4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:42.089484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:42.090062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:42.090348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:42.090623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:42.092258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:42.093774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:42.095446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 122/230] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:42.101286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:42.162132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c841592-f3e1-4c16-b6f1-1ba721543fe0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:42.171692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:42.198344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:42.198983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:42.199263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:42.199578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:42.200810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:42.202289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:42.203898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 109/231] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:41 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:42.213653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:42.306976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5208de0-e7a2-4477-a0c9-ac6160c7629c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:42.325419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca28db0-2808-4c6b-aa3a-06f9e46692f5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:42.360220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:42.360860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:42.361144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:42.361426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:42.362561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:42.364025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:42.372932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 110/232] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:42 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:42.373049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:42.373483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:42.373763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:42.374039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/1e680d0f82ac4865a20de5ea136f2dee/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:42.376069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, user_id=1e680d0f82ac4865a20de5ea136f2dee, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:42.377557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b85763-8135-4bc6-a385-6aaa19b1541e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:42.378492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:42.437064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:42.481100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 123/233] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:42 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/1e680d0f82ac4865a20de5ea136f2dee/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:42.512807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77848e22-8ddb-4081-8667-ab456642444f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:42.571005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:42.571694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:42.571974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:42.572250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/1e680d0f82ac4865a20de5ea136f2dee/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:42.574542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, user_id=1e680d0f82ac4865a20de5ea136f2dee, role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:42.576040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3c2a9-57ad-474c-86c3-2877271f1647 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:42.702886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 111/234] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:42 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/1e680d0f82ac4865a20de5ea136f2dee/roles/0bb62a6e624c40b4832f54f88761160b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:43.130508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfec4-8d6b-4932-8989-9f0ceb2139e8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.131018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfec4-8d6b-4932-8989-9f0ceb2139e8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.131089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfec4-8d6b-4932-8989-9f0ceb2139e8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.132984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 124/235] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:43.138405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a3c7-486f-4e17-a286-814a6c06d8ee None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.138907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a3c7-486f-4e17-a286-814a6c06d8ee None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.138981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a3c7-486f-4e17-a286-814a6c06d8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.162025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0774a3c7-486f-4e17-a286-814a6c06d8ee None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:43.163221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0774a3c7-486f-4e17-a286-814a6c06d8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:43.254913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 112/236] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:43.365177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c99f-0d7f-4159-b99d-b1fa0cc1c516 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.365490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c99f-0d7f-4159-b99d-b1fa0cc1c516 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.365740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c99f-0d7f-4159-b99d-b1fa0cc1c516 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.367914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 125/237] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:43.372974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a857-768e-400f-989b-d5589655c971 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.373265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a857-768e-400f-989b-d5589655c971 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.373541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a857-768e-400f-989b-d5589655c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.399102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6b07a857-768e-400f-989b-d5589655c971 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:43.402282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6b07a857-768e-400f-989b-d5589655c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:43.445710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:43.498924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 113/238] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:43.612690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95c99f-0d7f-4159-b99d-b1fa0cc1c516 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:43.634247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:43.757366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95c99f-0d7f-4159-b99d-b1fa0cc1c516 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:43.772398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07a857-768e-400f-989b-d5589655c971 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:43.816520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:43.817194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.817473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.817746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.842131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:43.846923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:43.847801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:43.848593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 126/239] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:43.855830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:43.940254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07a857-768e-400f-989b-d5589655c971 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:43.979342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:43.980325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:43.980847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:43.981348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:43.982950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:43.984792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:43.986585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 114/240] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:43.993766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:44.018270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.051103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.077948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cfd03-ace5-4819-9457-b4137d27f4ac None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:44.128170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:44.128814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.129092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.129827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.130503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:44.131202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:44.133397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675708ee-37d4-4e6b-875c-796855671489 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:44.146132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 127/241] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:44.148938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929d1d-698e-4804-9e40-b1893a300b4d None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.149391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929d1d-698e-4804-9e40-b1893a300b4d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.150068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34929d1d-698e-4804-9e40-b1893a300b4d None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.152844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 128/242] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:44.153805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929d1d-698e-4804-9e40-b1893a300b4d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:44.160408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd072c-4792-4d2c-bffc-6fce07d2d7f9 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:44.197132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:44.198123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.198590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.199047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.199878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:44.201832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f355-07f1-4d2b-8f3f-f80094db9101 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:44.275161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34929d1d-698e-4804-9e40-b1893a300b4d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:44.311145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:44.311835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.312116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.312392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] PATH_INFO: `/v3/users/d81fc15d197d4b9988083c09f9ef0998` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.313079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 115/243] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:43 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:35:44.314855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.315409 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.317591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.321889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.343194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:44.344950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:44.345083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] RBAC: Authorizing `identity:get_user(user_id=d81fc15d197d4b9988083c09f9ef0998)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:44.346561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:44.348565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 129/244] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:44 2022] GET /identity/v3/users/d81fc15d197d4b9988083c09f9ef0998 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:44.354469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:44.437753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 116/245] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:44.522530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fd145-b60b-4d8e-abcd-e7df482d4136 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:44.560094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:44.560725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.561004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.561271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.624238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.638433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:44.639908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435b9b-6a54-49b9-9e97-4476b8cea475 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:44.753340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 130/246] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:44.757506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:44.772131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb7902-f5f5-485f-b942-1a47922f7847 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.772439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb7902-f5f5-485f-b942-1a47922f7847 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.772710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb7902-f5f5-485f-b942-1a47922f7847 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.774579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 131/247] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:44.779520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.779797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.780064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.786061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.802034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:44.803434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:44.889996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 132/248] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:44.890789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:44.902075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b5e1-1865-4546-9c03-96f38780285b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:44.929128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:44.951776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:44.952120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:44.952369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:44.952725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:44.953889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:44.955333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:44.957847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 117/249] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:44 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:44.963877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.036244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c334f51-cd0a-4f2e-b221-ea7a9ed813a3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.077633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.078506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.078506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.078787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.088914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:45.092609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:45.097723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:45.103874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:45.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 133/250] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:45.105713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.111810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc8233-82fe-4e4c-bd79-60a50c117a86 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.160211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.160952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.161494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.161565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.162429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=d81fc15d197d4b9988083c09f9ef0998) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:45.162801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:45.165016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:45.171594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 118/251] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:44 2022] GET /identity/v3/role_assignments?user.id=d81fc15d197d4b9988083c09f9ef0998&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=273e4fba5ef141888990da15821d9dfb => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:45.207451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:45.228505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f301d-6bff-4a24-9dfd-0c7332d51597 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.266629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.267286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.267570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.267836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.268526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:45.268857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:45.270295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:45.280459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 134/252] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:45.286271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.341866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.482660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ae8aa-f96a-40f6-b860-64e57abde65a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.508097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a1ac3-9688-4d2d-88ce-8e36803fa1a0 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.526700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.527345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.527625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.527897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] PATH_INFO: `/v3/users/e6e7b3c03b534fd6813192545eaa442b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.529977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] RBAC: Authorizing `identity:get_user(user_id=e6e7b3c03b534fd6813192545eaa442b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:45.531390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423264d-47d4-4fd3-b4f3-79ce584019a2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:45.533198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 119/253] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:45 2022] GET /identity/v3/users/e6e7b3c03b534fd6813192545eaa442b => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:45.534172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8be2-68d9-4899-b733-501071bb52de None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.534493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8be2-68d9-4899-b733-501071bb52de None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.534758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8be2-68d9-4899-b733-501071bb52de None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.540676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 120/254] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:45.542147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.542431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.542805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.563597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:45.564878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:45.564988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.565860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.566334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.566798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.579873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:45.580711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:45.588835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:45.598611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 135/255] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:45.599419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.602165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:45.679436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 121/256] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:45.680301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.791728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aff77-fe5e-400d-9eb2-aaeef35b0038 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.840074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.841100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.841578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.842072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.843490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:45.853318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:45.855363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 136/257] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:45 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:45.861184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:45.920219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3ccf6-e54d-4eaa-b6d5-548b896f9402 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.938660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:45.972476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:45.973475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:45.973998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:45.974503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:45.975378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:45.976238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:45.978549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:45.994503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71d59-74d8-4df4-b15f-19c1fc452cf3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:45.996235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 122/258] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:46.006682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.024142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:46.074568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.075263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.075544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.075816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/e6e7b3c03b534fd6813192545eaa442b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.077817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, user_id=e6e7b3c03b534fd6813192545eaa442b, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:46.079293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:46.178081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e6f90-dd02-45f7-a77b-67a566fd162c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:46.216172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 137/259] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:45 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/e6e7b3c03b534fd6813192545eaa442b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:35:46.216757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.217005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.217161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.217498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.223135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:46.223135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:46.223352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.223438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 123/260] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:46.229607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.330456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:46.358843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf0e6-d4c4-4701-9703-7c2cbdbb15e2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:46.399147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.400215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.400496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.400845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.489605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ae02d-c7d3-4473-83b0-d87be4fd5364 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:46.492863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:46.495290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:46.534660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:46.541073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.541712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.541987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.542813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.558230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:46.563040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:46.565429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:46.567377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 138/261] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:46.572683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.593566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:46.621769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 124/262] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:46.631414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.797777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dc583-7ff5-4a9c-8328-b1f40be94480 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:46.817305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e5ccf-a93c-4b5a-afe7-5e9aed9b7e17 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:46.881450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.881986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.882262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.883161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.883896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:46.884198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:46.885648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:46.899031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 139/263] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:46.904678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:46.921414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:46.922428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:46.922933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:46.923413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:46.924865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:46.926520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:46.928696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 125/264] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:46.935411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:47.017028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:47.049871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:47.083917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c46a2-27ca-4061-b494-cd6bd940d58f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:47.141553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:47.143015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.143295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.143568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] PATH_INFO: `/v3/users/1e680d0f82ac4865a20de5ea136f2dee` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.144787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] RBAC: Authorizing `identity:get_user(user_id=1e680d0f82ac4865a20de5ea136f2dee)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:47.146251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:47.151219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 140/265] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/users/1e680d0f82ac4865a20de5ea136f2dee => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:47.153358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:47.270598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cabe-c041-4f71-a565-6db4f8e9e396 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:47.300008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c558ed0-1031-427a-9a14-2db2e39781a5 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:47.345770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:47.362241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:47.362999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.363279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.363546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.411838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:47.412462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.412761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.413031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.414311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=1e680d0f82ac4865a20de5ea136f2dee) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:47.414620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:47.418166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef6c4a-dc7c-46f5-bbef-820166429c8d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:47.440395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 126/266] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:46 2022] GET /identity/v3/role_assignments?user.id=1e680d0f82ac4865a20de5ea136f2dee&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:47.483721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:47.485257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:47.556386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497bc5-8c56-429b-ac3d-05510f9725cf None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.556746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497bc5-8c56-429b-ac3d-05510f9725cf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.556936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed497bc5-8c56-429b-ac3d-05510f9725cf None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.558610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 127/267] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:47.569547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed236-e8ee-45fc-9513-77ad3b629427 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.569907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed236-e8ee-45fc-9513-77ad3b629427 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.570127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ed236-e8ee-45fc-9513-77ad3b629427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.607816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d10ed236-e8ee-45fc-9513-77ad3b629427 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:47.609081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d10ed236-e8ee-45fc-9513-77ad3b629427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:47.667968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 141/268] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:47.674002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:47.718044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:47.744935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 128/269] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:47.768817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809c2e-54d3-4b60-b15e-708d19f2ce27 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.769127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809c2e-54d3-4b60-b15e-708d19f2ce27 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.769362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809c2e-54d3-4b60-b15e-708d19f2ce27 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.770702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 129/270] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:47.775719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.776000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.776276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.794616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:47.796018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:47.830149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d3890-c990-4f29-a121-75239a762b95 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:47.840458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:47.877906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 130/271] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:47.904751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:47.905680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:47.906148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:47.906612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:47.907895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:47.909534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:47.911315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 142/272] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:47 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:47.918107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.043372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8becbe-c36f-4f3f-8917-c7d2290de8f3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.083718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.084333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.084609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.084908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.085734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=1e680d0f82ac4865a20de5ea136f2dee) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:48.086043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.087563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790058c3-7b21-4f00-becd-c4fd6309a954 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.093627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 131/273] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:47 2022] GET /identity/v3/role_assignments?user.id=1e680d0f82ac4865a20de5ea136f2dee&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=0bb62a6e624c40b4832f54f88761160b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:48.101803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.188690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41532ab-fe5b-4543-8c5a-db9ce983fd91 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.189310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41532ab-fe5b-4543-8c5a-db9ce983fd91 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.189591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41532ab-fe5b-4543-8c5a-db9ce983fd91 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.191291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 132/274] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:48.202468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.203849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.204488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.239033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:48.242327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:48.264643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.281725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecf883-1750-4c52-833c-2f90b81d5851 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.301677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.331852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.333024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.333024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.333024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.347466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:48.354944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:48.355822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:48.356603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:48.358522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 143/275] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:48.359181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.366307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 133/276] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:48.366307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.491433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced76bf-3548-4e2d-9968-cd48554874ec None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.500879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359fc39e-57c6-4b38-a18a-b8e5993f96fb None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.529627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.530716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.531202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.531660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.532516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:48.533036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.534606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.544932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 134/277] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:48.547450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.548085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.548358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.548623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.550378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.559921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:48.566150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:48.567045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:48.567829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 144/278] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:48.574923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.591864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.592516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.672558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fb565-9aaa-4b9c-9f08-3127abfe7e3c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.695298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f44a7-633c-43a7-a707-4957a7a8c17a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.707348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.707980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.708258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.708552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] PATH_INFO: `/v3/users/3e320d75094c4d84abf17f008de42c11` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.722601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] RBAC: Authorizing `identity:get_user(user_id=3e320d75094c4d84abf17f008de42c11)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.724097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.725918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 135/279] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/users/3e320d75094c4d84abf17f008de42c11 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:48.726545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.736520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.737180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.737460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.737732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.738407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:48.738711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.740095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.749269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 145/280] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:48.749936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.823363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.827949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:48.842088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8db67d-aa4f-4ea8-8a39-0416c2a9919f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.857019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032f4b-fb9e-4f14-8c0f-c096f9457a11 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:48.874316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.874936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.875209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.875474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.876620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.878254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.879803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 136/281] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:48.880553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:48.889887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:48.890521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.890808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.891082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.891805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, user.id=3e320d75094c4d84abf17f008de42c11, scope.system=all) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:48.892109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:48.893570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765600b-c947-4c6b-b4ec-0b65237fc100 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:48.899141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 146/282] 10.222.0.72 () {56 vars in 1308 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/role_assignments?user.id=3e320d75094c4d84abf17f008de42c11&scope.system=all&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:48.900122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4602d-83ac-46a9-ae22-92903cce6674 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:48.900401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4602d-83ac-46a9-ae22-92903cce6674 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:48.900700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4602d-83ac-46a9-ae22-92903cce6674 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:48.911198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 147/283] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:48.911198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4602d-83ac-46a9-ae22-92903cce6674 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.005159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fe32-b2eb-43ec-b37d-ea38982d57a7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.028044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4602d-83ac-46a9-ae22-92903cce6674 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.058160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.058803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.059082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.059350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] PATH_INFO: `/v3/users/d81fc15d197d4b9988083c09f9ef0998` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.060902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d81fc15d197d4b9988083c09f9ef0998)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.062033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634fb45-d53b-4f7f-9150-216fcb3ba0f0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.063746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 137/284] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:48 2022] GET /identity/v3/users/d81fc15d197d4b9988083c09f9ef0998 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:49.064672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.064963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.065235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.071249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.071985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.072258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.072644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.073320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.074756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.083218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:49.084401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:49.084594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:49.099610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:49.151753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 138/285] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:49.152598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.194216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 148/286] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:35:48 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:49.279858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcfb28-59de-4acd-8542-1e0f317a4e78 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.318759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.319389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.319731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.320559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.321824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.323307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.325089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 139/287] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:49 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:49.331168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.426688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.452386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:49.456254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b1aed-8116-4dd2-bfb3-e25458f3ad26 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.503114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.503800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.504082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.504361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/d81fc15d197d4b9988083c09f9ef0998/roles/273e4fba5ef141888990da15821d9dfb` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.506357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, user_id=d81fc15d197d4b9988083c09f9ef0998, role_id=273e4fba5ef141888990da15821d9dfb)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.508263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.573684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325cc1-ab63-4514-ab8e-d8de9ad46e75 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.586289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 149/288] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:35:49 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/users/d81fc15d197d4b9988083c09f9ef0998/roles/273e4fba5ef141888990da15821d9dfb => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:49.616065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.617422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.617708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.617978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.630552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:49.635532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:49.636478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:49.637325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:49.640757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 140/289] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:49.643141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.708997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:49.763044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cf69c-d850-44e3-859b-e7f88dfad744 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.801377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.802037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.802321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.802590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.803265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:49.803627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.805133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.818769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 150/290] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:49.820150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:49.910387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:49.935909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76bfb3-fa8b-47e8-b643-27c82159ef5f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:49.980517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:49.981191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:49.981469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:49.981738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] PATH_INFO: `/v3/users/e6e7b3c03b534fd6813192545eaa442b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:49.985333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] RBAC: Authorizing `identity:get_user(user_id=e6e7b3c03b534fd6813192545eaa442b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:49.986798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:49.988769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 141/291] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:49 2022] GET /identity/v3/users/e6e7b3c03b534fd6813192545eaa442b => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:49.995276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:50.171659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5d41f-7090-47c0-835c-0bac5db5942a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:50.193682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:50.223000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:50.223631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:50.223909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:50.224184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:50.309652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:50.311137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:50.461650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 151/292] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:50.467395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:50.612462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91779ce-9b4f-48cc-9598-f8e27ecf7c22 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:50.658401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:50.659035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:50.659315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:50.659583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:50.660721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:50.662160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dbb9e-307c-48c8-a39f-e9830d36fb29 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:50.663808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 142/293] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:50 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:50.669934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:50.700697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:50.831103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3049a-44ea-48e3-9dd1-51c13c733b6f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:50.872265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:50.872923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:50.873204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:50.873476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:50.874221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=e6e7b3c03b534fd6813192545eaa442b) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:50.874525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:50.875976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e2f3-7c85-47eb-b03d-69c65912bc00 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:50.885527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 152/294] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:50 2022] GET /identity/v3/role_assignments?user.id=e6e7b3c03b534fd6813192545eaa442b&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:51.261646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638302c5-d014-4c23-a6ff-04b0116e85b9 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.261646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638302c5-d014-4c23-a6ff-04b0116e85b9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.261646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638302c5-d014-4c23-a6ff-04b0116e85b9 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.262890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 143/295] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:51.267939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfc104-dbe2-4edb-bcc7-49f93a2257e6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.268224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfc104-dbe2-4edb-bcc7-49f93a2257e6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.268499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfc104-dbe2-4edb-bcc7-49f93a2257e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.291053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f3dfc104-dbe2-4edb-bcc7-49f93a2257e6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:51.292259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f3dfc104-dbe2-4edb-bcc7-49f93a2257e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:51.368515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 153/296] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:51.677542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638302c5-d014-4c23-a6ff-04b0116e85b9 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:51.696411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:51.723610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaf5dc-0784-4c38-9544-cc732bba201c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.724548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaf5dc-0784-4c38-9544-cc732bba201c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.725079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eaf5dc-0784-4c38-9544-cc732bba201c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.727187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 154/297] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:51.738338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105ced-0354-4bbb-8403-317e1f675a19 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.738338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105ced-0354-4bbb-8403-317e1f675a19 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.738338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105ced-0354-4bbb-8403-317e1f675a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.755965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-34105ced-0354-4bbb-8403-317e1f675a19 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:51.757225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-34105ced-0354-4bbb-8403-317e1f675a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:51.759204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:51.817346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638302c5-d014-4c23-a6ff-04b0116e85b9 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:51.836734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 155/298] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:51.863450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:51.864139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.864421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.864714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.875868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:51.876198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980764-916b-401a-b977-ac7580eb0dc7 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.876713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980764-916b-401a-b977-ac7580eb0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.877203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e980764-916b-401a-b977-ac7580eb0dc7 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.878792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 156/299] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:51.887003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:51.887887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:51.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 144/300] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:51.889360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df919-2c7e-4c51-9bc2-2409ae12a35a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:51.889360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df919-2c7e-4c51-9bc2-2409ae12a35a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:51.889360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df919-2c7e-4c51-9bc2-2409ae12a35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:51.890103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:51.911126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-625df919-2c7e-4c51-9bc2-2409ae12a35a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:51.912838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-625df919-2c7e-4c51-9bc2-2409ae12a35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:51.973112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:51.997613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec01f87-8c7d-4130-a804-b2aebc68fa32 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.004861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 157/301] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:52.009212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9b3a4-3207-4fc9-be51-aad568094cca None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.009212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9b3a4-3207-4fc9-be51-aad568094cca None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.009212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9b3a4-3207-4fc9-be51-aad568094cca None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.009212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 158/302] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:52.015153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.015429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.015702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.037481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:52.037940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.038742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:52.039049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.039554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.040038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.040981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:52.041495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.042984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.051571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 145/303] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:52.057888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.119154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.132676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 159/304] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:52.132676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.199376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5cbc9-e253-4303-abe1-fe51fce5f3f5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.239278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c4f07-b907-4233-8da7-515d60f901ea None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.245530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.246249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.246763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.247672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] PATH_INFO: `/v3/users/3e320d75094c4d84abf17f008de42c11` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.248962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] RBAC: Authorizing `identity:get_user(user_id=3e320d75094c4d84abf17f008de42c11)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.251313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.256807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 146/305] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/users/3e320d75094c4d84abf17f008de42c11 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:52.259936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.265430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.283261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.283894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.284166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.284431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.294962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:52.299696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:52.300961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:52.301858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:52.306324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 160/306] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:52.307058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.365683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.375600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a59c7-5e50-4096-bf76-7e719eeb958d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.415045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.415490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64391b2-f256-44ca-9c74-7aca1070d1d3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.415696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.415968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.416236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] PATH_INFO: `/v3/system/users/3e320d75094c4d84abf17f008de42c11/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.417663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3e320d75094c4d84abf17f008de42c11, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.419037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.449923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.450547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.450819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.451082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.452171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.453612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.455162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 161/307] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:52.455916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.513321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 147/308] 10.222.0.72 () {58 vars in 1334 bytes} [Wed Aug 24 07:35:52 2022] PUT /identity/v3/system/users/3e320d75094c4d84abf17f008de42c11/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:52.514285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.561886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec2e69-6d62-464e-8b68-2f915e2b1066 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.592978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.595233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.604693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.605318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.605666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.605855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.606527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:52.606824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.608221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.622941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 162/309] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:52.622941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.644865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392c329-5abb-4d3b-9151-8d2da473cbd1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.686729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.687388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.687645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.687894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.699476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:52.704904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:52.705786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 148/310] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:52.707418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.735599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c730c-e3e4-4969-ad11-1959517007d7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.774401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.774938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.775215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.775484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] PATH_INFO: `/v3/groups` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.776111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.777645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.818057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dfd2f-9f3c-4b7e-af1d-54c5bbed6e22 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:52.844110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.852490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:52.853163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:52.853427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:52.853695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] PATH_INFO: `/v3/users/ffde9676676340158f712adea638a625` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:52.861535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 163/311] 10.222.0.72 () {60 vars in 1134 bytes} [Wed Aug 24 07:35:52 2022] POST /identity/v3/groups => generated 229 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:52.868204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] RBAC: Authorizing `identity:get_user(user_id=ffde9676676340158f712adea638a625)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:52.869846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:52.869947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.871734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 149/312] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/users/ffde9676676340158f712adea638a625 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:52.877936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:52.945258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:52.999184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a9c0e-1746-421c-b87b-6d88f3d17ea2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.020756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4c35c-7f78-4bc7-811b-658fc12fb6a6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.044098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.044640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.044943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.045212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.046009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, user.id=ffde9676676340158f712adea638a625, scope.system=all) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:53.046323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.047783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.054864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 150/313] 10.222.0.72 () {56 vars in 1308 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/role_assignments?user.id=ffde9676676340158f712adea638a625&scope.system=all&role.id=273e4fba5ef141888990da15821d9dfb => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:53.071466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.072105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.072542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.073028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.073753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:53.074051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.076104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.088758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 164/314] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:53.094197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:53.156126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:53.211811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e50101-5f3e-4731-b498-c0bc1535759c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.248111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.248779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.249060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.249329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] PATH_INFO: `/v3/users/d81fc15d197d4b9988083c09f9ef0998` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.250488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] RBAC: Authorizing `identity:get_user(user_id=d81fc15d197d4b9988083c09f9ef0998)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.251951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.253939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 151/315] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:53 2022] GET /identity/v3/users/d81fc15d197d4b9988083c09f9ef0998 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:53.259959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:53.371829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd3b63-8c39-47ab-b522-6b04c1957197 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.384425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:53.409366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.410081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.410417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.410694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.482286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.483760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.587975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 165/316] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:35:53.593757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:53.702204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab467c0c-f7c2-42a9-b764-ca0c48547df5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.722255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:53.740940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.741887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.742363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.742830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.744228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.745944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751de231-32e1-4771-8872-1e17ae6767cd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.748064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 152/317] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:53 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:53.754159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:53.767695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:53.862696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cf66-8549-425f-acb4-e0386bda2336 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:53.897983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:53.898593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:53.898869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:53.899139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:53.899870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, scope.project.id=4c677764ecb74305a5ec39c26cce1a27, user.id=d81fc15d197d4b9988083c09f9ef0998) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:53.900173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:53.901653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789e573-97c9-4e04-83ec-ba6d4c1ce751 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:53.908441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 166/318] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:35:53 2022] GET /identity/v3/role_assignments?user.id=d81fc15d197d4b9988083c09f9ef0998&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=273e4fba5ef141888990da15821d9dfb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:54.494312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e7c46-2663-4a54-8d39-36f45b196d82 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.494630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e7c46-2663-4a54-8d39-36f45b196d82 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.494867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e7c46-2663-4a54-8d39-36f45b196d82 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.496345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 153/319] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:54.501527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357e0-878c-4317-943d-504439a0807f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.501814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357e0-878c-4317-943d-504439a0807f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.502081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1357e0-878c-4317-943d-504439a0807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.520433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-bb1357e0-878c-4317-943d-504439a0807f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:54.521661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bb1357e0-878c-4317-943d-504439a0807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:54.591347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 167/320] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:54.851447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5062-a62c-47cf-a5e9-9f879a658658 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.851990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5062-a62c-47cf-a5e9-9f879a658658 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.851990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5062-a62c-47cf-a5e9-9f879a658658 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.853351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 154/321] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:54.858422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:54.858813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:54.859008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:54.860988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d5062-a62c-47cf-a5e9-9f879a658658 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:54.865967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:54.877196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:54.878369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:54.955050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 168/322] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:54.980738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d5062-a62c-47cf-a5e9-9f879a658658 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.028413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.029256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.029457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.029682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.041433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-df6ac6f2-84e3-432d-9b12-a5e030439524 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:55.046304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:55.047191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:55.048445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:55.052315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 155/323] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:35:55.053676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.064817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda59e-9061-43f8-b8d3-d0f6395e428e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.065108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda59e-9061-43f8-b8d3-d0f6395e428e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.066118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dda59e-9061-43f8-b8d3-d0f6395e428e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.068701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 156/324] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:35:55.072470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.073050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.073481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.093519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:55.094721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:55.142990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.168278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21abe4-ae90-408e-a9f9-ffe9f42d4b18 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.172699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 157/325] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:35:55.206147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.206760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.207035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.207305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.207977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.208278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.210278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.211868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.219347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 169/326] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.224896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.225044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.354812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a380f-5422-4904-8f89-935e37e44afb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.357875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0e4c4-23ca-4ffa-ab4d-90a181d19df2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.394073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.394627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.394893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.395164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] PATH_INFO: `/v3/users/3e320d75094c4d84abf17f008de42c11` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.395772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.396493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] RBAC: Authorizing `identity:get_user(user_id=3e320d75094c4d84abf17f008de42c11)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.396600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.396855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.397195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.398393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.399044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.402897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 170/327] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/users/3e320d75094c4d84abf17f008de42c11 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.403059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.406799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 158/328] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:35:55.410721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.410862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.462897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.482197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.517610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa3ed0-4c5f-4adf-b553-1cd5db331ad0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.519882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc9ff08-d4cd-4bf0-b55d-4e3a3a021971 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.550742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.551347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.551626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.551896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.552626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] RBAC: Adding query filter params (role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, user.id=3e320d75094c4d84abf17f008de42c11, scope.system=all) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.552968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.554960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.558340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.559026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.559225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.559497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] PATH_INFO: `/v3/groups` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.560121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.561581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.566911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 159/329] 10.222.0.72 () {56 vars in 1308 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/role_assignments?user.id=3e320d75094c4d84abf17f008de42c11&scope.system=all&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 510 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.567870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.668338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 171/330] 10.222.0.72 () {60 vars in 1134 bytes} [Wed Aug 24 07:35:55 2022] POST /identity/v3/groups => generated 226 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:35:55.703243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e63eb-16af-4a4f-a4d1-ecafed557ce1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.705617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.744089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.745401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.745682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.745956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.759948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:55.766176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:55.767109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:55.767924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 160/331] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:55.774127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:55.881022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f046ceb-c566-48c5-862b-56a1dc494789 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:55.905509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:55.917337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:55.917948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:55.918219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:55.920220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:55.920220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:55.920220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:55.920825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:55.931964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 172/332] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:55.937888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.046492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6f282-540f-4c48-8130-882a0bee8a7a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.082029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.083058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.083334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.083600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] PATH_INFO: `/v3/users/ffde9676676340158f712adea638a625` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.085259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] RBAC: Authorizing `identity:get_user(user_id=ffde9676676340158f712adea638a625)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.086760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd96ab-318c-4e6b-b98e-e764743f2111 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.088543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 161/333] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:35:55 2022] GET /identity/v3/users/ffde9676676340158f712adea638a625 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:56.094423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:56.203138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02d1a-c9e1-4861-a256-287731e8d3fc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:56.239593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:56.240544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:56.241018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:56.241537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] PATH_INFO: `/v3/system/users/ffde9676676340158f712adea638a625/roles/273e4fba5ef141888990da15821d9dfb` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:56.243137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ffde9676676340158f712adea638a625, role_id=273e4fba5ef141888990da15821d9dfb)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:56.244676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba995682-d999-4459-8646-0600ca150b55 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:56.358940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 173/334] 10.222.0.72 () {58 vars in 1334 bytes} [Wed Aug 24 07:35:56 2022] PUT /identity/v3/system/users/ffde9676676340158f712adea638a625/roles/273e4fba5ef141888990da15821d9dfb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:35:58.035503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef036c1f-54a2-498c-be5a-23e0afc1eb5b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.036061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef036c1f-54a2-498c-be5a-23e0afc1eb5b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.036061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef036c1f-54a2-498c-be5a-23e0afc1eb5b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.037699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 162/335] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:58.042998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea811aa-62f7-499b-8020-42c91fe9569a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.043831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea811aa-62f7-499b-8020-42c91fe9569a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.044304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea811aa-62f7-499b-8020-42c91fe9569a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.069568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6ea811aa-62f7-499b-8020-42c91fe9569a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:58.071092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6ea811aa-62f7-499b-8020-42c91fe9569a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:58.125262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:58.148289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 174/336] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:58.318011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130748-b2b5-4a12-aeab-b40ebf468d73 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.318649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130748-b2b5-4a12-aeab-b40ebf468d73 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.319151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e130748-b2b5-4a12-aeab-b40ebf468d73 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.320968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 163/337] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:58.325960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.326246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.326522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.348830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:58.350031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:58.361713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f48713-0db6-46e2-a372-b1e706d2bdb4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.361991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f48713-0db6-46e2-a372-b1e706d2bdb4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.362266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f48713-0db6-46e2-a372-b1e706d2bdb4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.363875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 164/338] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:58.369177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.369454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.369724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.405670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:58.406868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:58.470035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 175/339] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:58.478788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:58.549264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 165/340] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:35:58.608138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:58.734616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97da4d-c9b3-472c-81ae-c44a71218d59 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:58.745707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:58.775429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:58.776070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.776343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.776608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.790964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:58.796059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:58.802800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:58.804148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 176/341] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:58.808870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:58.927193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77140cea-9550-4af5-a84f-c0d2f15f497e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:58.975330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:58.975964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:58.976238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:58.976502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:58.980736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:58.980736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:58.980736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:58.992872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 166/342] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:58.999034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.114135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.137524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21b242-b00d-4903-a5c7-6bf205d1f33e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.181987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.183052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.184372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.185146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.189629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.258961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.317503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.320673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.336676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4994129-2f98-42de-aff2-1286285a5cab None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.452724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.453310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.453588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.453861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.459807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 177/343] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:35:59.460924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.474300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:59.480405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:59.483251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 167/344] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:59.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.570069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.587578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.609657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0bd91-4e88-4227-b157-584e4191f034 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.619418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c00b-1c1f-4131-96f8-996481d2053c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.648741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.649366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.649643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.649914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.660838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:35:59.666478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:35:59.667472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 178/345] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:35:59.668434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.676229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.676908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.677187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.677462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.679235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.681028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eef25-998c-4e38-8cc8-94fcf49b7eb9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.683515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 168/346] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:59.687912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929467-e18a-4290-bf4b-d03f7db48fc3 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.687912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929467-e18a-4290-bf4b-d03f7db48fc3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.687912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929467-e18a-4290-bf4b-d03f7db48fc3 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.688567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 169/347] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:59.694517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e382-0496-492e-adf7-5eef5184f655 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.694809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e382-0496-492e-adf7-5eef5184f655 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.695084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e382-0496-492e-adf7-5eef5184f655 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.696400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 170/348] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:35:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:35:59.697141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e382-0496-492e-adf7-5eef5184f655 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.791663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e8764-94a9-4163-ae29-e229625f762f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.831041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e382-0496-492e-adf7-5eef5184f655 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:35:59.835903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:35:59.844095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.844822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.845083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.845353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.846027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:59.846328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.861954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.867215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 179/349] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:59.868246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:35:59.876030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:35:59.876669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.876952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.877228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.877902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:35:59.878205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:35:59.880130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db848f93-3c4b-4568-9c85-a6b7a0a01277 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:35:59.893568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 171/350] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:35:59.894900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b46ff-2ff5-4e0c-9e12-9306e23dbc66 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:35:59.895184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b46ff-2ff5-4e0c-9e12-9306e23dbc66 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:35:59.895456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b46ff-2ff5-4e0c-9e12-9306e23dbc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:35:59.920589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5c3b46ff-2ff5-4e0c-9e12-9306e23dbc66 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:35:59.926191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5c3b46ff-2ff5-4e0c-9e12-9306e23dbc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:35:59.938280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.002724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f81e0-60ec-4cb9-9dea-9d17951a92b3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.017592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 172/351] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:00.019384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.019677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.019950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.041284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.042027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.042250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.042514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.043255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:00.044573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:00.049696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.051158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.053948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 180/352] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:35:59 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.054625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.122653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.131837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 173/353] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:00.132683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.177584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a33e2-e54d-477a-9e96-ea0dba59c059 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.234750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.235369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.235649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.235920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.254946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.297650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c62a38-21f8-4d1d-9c2f-b4533f3ff79e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.322702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.324611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10872ac8-7f0f-4ef1-8355-aed6f1e144fc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.349364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.350043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.350830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.351095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.392041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.443085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.444587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.453214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 181/354] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:00.454317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003735ac-edb0-4237-b6c5-dc511ee1de69 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.454667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003735ac-edb0-4237-b6c5-dc511ee1de69 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.454945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003735ac-edb0-4237-b6c5-dc511ee1de69 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.456193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 182/355] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:00.457022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003735ac-edb0-4237-b6c5-dc511ee1de69 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.532855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.568933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 174/356] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:00.569982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.579455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003735ac-edb0-4237-b6c5-dc511ee1de69 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.619750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.620366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.620661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.620946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.621734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:00.622429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.623430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.630269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 183/357] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.631845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.660867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.699174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02d2c1-b5a5-4218-8adb-e8a335708f03 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.754476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.754476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.754737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.755040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.769433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:00.775290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:00.776168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:00.777420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 175/358] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:00.778304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.779831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0d5d0-51ca-4808-828c-3ccccdb70773 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.816332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:00.835117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.835117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.835117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.835117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.845586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1825b6ed-f35e-4130-9883-8f5ad53b0221 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:00.850698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:00.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:00.852416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 184/359] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:00.853172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.853735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.853735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.875810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:00.877128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:00.904611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7ba97-53aa-4f0f-8be0-5e0fdc488455 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:00.951044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:00.951692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:00.951973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:00.952261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:00.955825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:00.957598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:00.960593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 176/360] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:00.962066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 185/361] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:00.962297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.962933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:00.979450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.088817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984bafa-5ce7-4900-aaf5-aa646afb2de5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.103047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.107204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49006d-6e54-4c15-a521-52b78fc4e249 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.132537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.133314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.133687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.134059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.134833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:01.135233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.138094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.149368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 186/362] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.152678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.157529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.158683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.159133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.159550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.163278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.165797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.168001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 177/363] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:00 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.169831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.245458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.288156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adfd7d-cf7c-4a98-85ec-5916fffa4bd4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.320843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060bc7b-0949-47ef-891f-491f8a5132d2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.332986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.333979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.334943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.335344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.336148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:01.336599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.338555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.349383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 187/364] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.350560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.369511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.370152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.370430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.370705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.371811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.374053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.374903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 178/365] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.375667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.425732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.495668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6e61-b530-4fda-934b-5f7634bf0962 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.501575 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882cc6-f41c-4138-aaae-73ee3f8ce088 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.539283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.539904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.540182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.540509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.543350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.545643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.546505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.546956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.547400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.548711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.550323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.553349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 179/366] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.554358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.632400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.633903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.683875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed6416-5332-47a1-b969-e03685e41a87 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.687364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.734554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.735196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.735475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.735744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.749990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:01.754776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:01.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:01.757325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 180/367] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:01.758084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.761727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 188/368] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:01.762775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.801799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:01.876416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356833d-1e52-43d3-91a3-c73fe96ab92e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.892942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff0b5-93d1-4019-a2d7-ae7b41b3a964 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:01.919188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.919828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.920104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.920372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] PATH_INFO: `/v3/users/ffde9676676340158f712adea638a625` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.924149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] RBAC: Authorizing `identity:get_user(user_id=ffde9676676340158f712adea638a625)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.924346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.926291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 181/369] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/users/ffde9676676340158f712adea638a625 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.927182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.937606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:01.938216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:01.938499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:01.938773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:01.939506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:01.939810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:01.941321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:01.947238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 189/370] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=0bb62a6e624c40b4832f54f88761160b => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:01.947904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:01.978067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.077524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e7ad6-ae11-4c98-9436-14bb4bb33a2f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.083804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.088681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6152cc1-808f-4188-9248-4ea60c6bd592 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.127907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.128513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.128882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.129216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.129946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:02.130255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.132254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.136120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.137873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.137873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.137873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.138131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:02.138327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.139740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.140165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 182/371] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.145072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.154037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 190/372] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.154856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.278617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.286501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89096d7d-2a9e-4717-9e93-f0183f9a6d57 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.298721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf9203-7df1-437a-b4e6-0cf39edaa763 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.335361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.336401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.336913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.337378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.338904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.340636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.343053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 183/373] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.344091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.400418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.400989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.401268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.401540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.402286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] RBAC: Adding query filter params (role.id=273e4fba5ef141888990da15821d9dfb, user.id=ffde9676676340158f712adea638a625, scope.system=all) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:02.404357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.405902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.452688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 191/374] 10.222.0.72 () {56 vars in 1308 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/role_assignments?user.id=ffde9676676340158f712adea638a625&scope.system=all&role.id=273e4fba5ef141888990da15821d9dfb => generated 510 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.452688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.471989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054723bd-1177-4e73-b4d5-30caad7c6ffa None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.506663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.518898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.520016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.520488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.520979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.607237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.653921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.655734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.705516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8477ab-1d79-4b43-810f-21f1519f7d69 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.781992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 184/375] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:02.785963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.786868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.787922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.788119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.788574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.789950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.791615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.797850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 192/376] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.803388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:02.865587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.909217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba95a5-cf65-47db-8ad1-f1d30e4a78bf None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.954011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:02.958014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:02.958949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.959780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.960053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] PATH_INFO: `/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.961735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] RBAC: Authorizing `identity:get_project(project_id=8d87d471ae2b4849a0dda41c93ab2dfd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:02.963796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9af92-6231-483f-808e-1a77fdc81c72 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:02.966610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 185/377] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:02.968437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbd5b5-bc00-46ac-84b5-d6b17f91a9b1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:02.968866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbd5b5-bc00-46ac-84b5-d6b17f91a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:02.969139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbd5b5-bc00-46ac-84b5-d6b17f91a9b1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:02.970381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb484-77ae-4b21-909d-ec2564b5fd71 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:02.973911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 186/378] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:02.977497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbd5b5-bc00-46ac-84b5-d6b17f91a9b1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.020812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.022165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.022462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.023043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.024251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:03.024865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.026307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342b665-0bc6-4740-9bff-a6c4c5484a28 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.036792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 193/379] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=0bb62a6e624c40b4832f54f88761160b => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:03.038213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.038492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.040986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.079271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:03.080536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:03.171654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbd5b5-bc00-46ac-84b5-d6b17f91a9b1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.234593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 194/380] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:03.239534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.244674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.245326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.245605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.245879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] PATH_INFO: `/v3/groups/853c78aab5754b7cab42d2a242568d55` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.252158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] RBAC: Authorizing `identity:get_group(group_id=853c78aab5754b7cab42d2a242568d55)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.253641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.257842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 187/381] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:02 2022] GET /identity/v3/groups/853c78aab5754b7cab42d2a242568d55 => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:03.261985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.353065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:03.373694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff2752-e129-4d26-ae33-94ba40070bee None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.411882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.413106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.413383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.413653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.414391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] RBAC: Adding query filter params (group.id=853c78aab5754b7cab42d2a242568d55, role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=8d87d471ae2b4849a0dda41c93ab2dfd) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:03.414695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:03.416144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:03.422587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 195/382] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/role_assignments?group.id=853c78aab5754b7cab42d2a242568d55&scope.project.id=8d87d471ae2b4849a0dda41c93ab2dfd&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:03.661965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.783879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb75cb-8ceb-4b5e-90d2-380bbafda860 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:03.829298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:03.829943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:03.830223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:03.830490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:03.845515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:03.850895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:03.852925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:03.854834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 188/383] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:03.858305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:03.972623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583d69d-75dc-4c17-b61a-a73808e00b64 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.008607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.009248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.009619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.009892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.010629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:04.010933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.012323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.021368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 196/384] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:36:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:04.027622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.101221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.155435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16351bac-a8f4-44da-baf4-c1e29c636812 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.268688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.268688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.268688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.268688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.269169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.269169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820a89f-3820-490c-80e2-31d9808c7cd7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.269822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 189/385] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:04.284675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.296672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.447428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d48d-65c7-4e89-b3c1-cd17e9722633 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.467194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b5a5-fe74-4267-b9ac-474b6eeb638f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.467473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b5a5-fe74-4267-b9ac-474b6eeb638f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.467741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065b5a5-fe74-4267-b9ac-474b6eeb638f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.469367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 190/386] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:04.477586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b3d5f-929b-40a3-96e4-87d9e08fd0f2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.477862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b3d5f-929b-40a3-96e4-87d9e08fd0f2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.478135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b3d5f-929b-40a3-96e4-87d9e08fd0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.505229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b59b3d5f-929b-40a3-96e4-87d9e08fd0f2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:04.506458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b59b3d5f-929b-40a3-96e4-87d9e08fd0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:04.510780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.511420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.511695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.511966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.513816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.515414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23557a8-f5d3-4acf-ade5-93477424156b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.519111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 197/387] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:04 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:04.520251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361103d-6800-4f3a-8e78-ece7688f0e83 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.520528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361103d-6800-4f3a-8e78-ece7688f0e83 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.527562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361103d-6800-4f3a-8e78-ece7688f0e83 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.527562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 198/388] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:04.528503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361103d-6800-4f3a-8e78-ece7688f0e83 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:04.612796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.658646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361103d-6800-4f3a-8e78-ece7688f0e83 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:04.676683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 191/389] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:04.676683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.676683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.676683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.713668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:04.714873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:04.731269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:04.732217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:04.732635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:04.733121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/groups/89271033f16444eb9fcc320c5df6a3b8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:04.737453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, group_id=89271033f16444eb9fcc320c5df6a3b8, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:04.739086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:04.814349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:04.821762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 199/390] 10.222.0.72 () {58 vars in 1442 bytes} [Wed Aug 24 07:36:04 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/groups/89271033f16444eb9fcc320c5df6a3b8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:04.833321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 192/391] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:05.011632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.123673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a1109-2c45-41a5-8452-4772a49aab07 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.169465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.170114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.170391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.170662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.172813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:05.176472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.184668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:05.190272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:05.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:05.192102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 200/392] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:05.198799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.305959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03472626-a732-4db4-a2e9-0a3d0dda7ef1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.315810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e33060-1ae2-4bf6-99ec-1c2577dcc86f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.345962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.346986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.347464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.347927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.358003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.358609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.358884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.359154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.359824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:05.360125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.361534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.371652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:05.386400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:05.387328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:05.388127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 193/393] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:05.389528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 201/394] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:05.393860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.398329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.445018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:05.522041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:05.530470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898dc2c-e8d9-4d6d-a68b-746f0c092c0d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.568858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.569514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.569795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.570062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.571813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.573774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.576407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 194/395] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:05.582580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.601311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9dab-500a-43ca-bdf6-fac63dc21cfd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.688490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.689661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.689939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.690274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.691486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:05.691856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.693974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.704141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 202/396] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:05.709959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.738807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361a1b-e326-4306-b56b-db68c1e29375 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.755534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:05.786083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.786721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.791067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.791372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.793616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.795071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.797827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 195/397] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:05.806262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.888157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457e507-3b59-4512-8925-a7ba68c6b221 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:05.910669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:05.936071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:05.936744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.937024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.937291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.938470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:05.940692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50757712-1c98-4e36-8efc-c1a3d67e642c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:05.942579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 203/398] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:05.943851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5858b-7783-42f7-8b5e-9750164b1fa3 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:05.944129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5858b-7783-42f7-8b5e-9750164b1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:05.944394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5858b-7783-42f7-8b5e-9750164b1fa3 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:05.945621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 204/399] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:05.950865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5858b-7783-42f7-8b5e-9750164b1fa3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:05.963377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074541f3-ed7a-45e3-b32f-5e96c9956e48 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.000843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.001515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.001797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.002073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/groups/89271033f16444eb9fcc320c5df6a3b8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.004322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, group_id=89271033f16444eb9fcc320c5df6a3b8, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:06.006190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03146cf-7f0e-4d7c-b310-6cb866bfd058 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:06.057518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5858b-7783-42f7-8b5e-9750164b1fa3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.096699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 196/400] 10.222.0.72 () {58 vars in 1442 bytes} [Wed Aug 24 07:36:05 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/groups/89271033f16444eb9fcc320c5df6a3b8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:36:06.096699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54423a3-ae7a-4da0-8da2-3fd52f16dc30 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.096699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54423a3-ae7a-4da0-8da2-3fd52f16dc30 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.096699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54423a3-ae7a-4da0-8da2-3fd52f16dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.100273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.100925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.101295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.101624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.102870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:06.104320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78134dc7-3401-4eca-892e-eaa0f763b168 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:06.106704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 205/401] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:05 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:06.107961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591613-503d-4a79-b68b-13497cc87800 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.108237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591613-503d-4a79-b68b-13497cc87800 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.108504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591613-503d-4a79-b68b-13497cc87800 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.110206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 206/402] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:06.113336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd591613-503d-4a79-b68b-13497cc87800 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:06.118332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e54423a3-ae7a-4da0-8da2-3fd52f16dc30 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:06.119595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e54423a3-ae7a-4da0-8da2-3fd52f16dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:06.165956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:06.202158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:06.217067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 197/403] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:06.221128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.221442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.221721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.259676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:06.261116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:06.283124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd591613-503d-4a79-b68b-13497cc87800 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.335450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.336395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.336765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.337063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/groups/89271033f16444eb9fcc320c5df6a3b8/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.341944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f3e29dc1cb740dcae1432624787cdb3, group_id=89271033f16444eb9fcc320c5df6a3b8, role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:06.343453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:06.345454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 198/404] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:06.431026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 207/405] 10.222.0.72 () {58 vars in 1442 bytes} [Wed Aug 24 07:36:06 2022] PUT /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3/groups/89271033f16444eb9fcc320c5df6a3b8/roles/0bb62a6e624c40b4832f54f88761160b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:36:06.521950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:06.663145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:06.666744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216bc8b-cb6b-404a-8882-e335414fb761 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.667466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:06.726277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.726857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.727137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.727410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.750222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:06.755072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:06.756604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:06.757525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 199/406] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:06.763822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:06.807711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192cacb-8d0b-4315-ab25-4bcb83bbbf47 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.822223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:06.843162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.848553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.848553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.848553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.857328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:06.863261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:06.864175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:06.865798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 208/407] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:06.876016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:06.893262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af224396-b6c7-41e8-8e59-1276717efd03 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:06.964129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:06.964778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:06.965057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:06.965331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:06.966012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:06.966320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:06.967698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:06.985532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 200/408] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:06.995704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.016492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b5847-a69d-42f6-9fd1-898d4feb984d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.054337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.057468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.058348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.058836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.059305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.060335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:07.060886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.062358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe812fc-2f70-4a59-9bad-040f6cc8ea3e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.071175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 209/409] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.078107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d1dc-d468-460c-8029-edc536a4e381 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.078107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d1dc-d468-460c-8029-edc536a4e381 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.078107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798d1dc-d468-460c-8029-edc536a4e381 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.078107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 210/410] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:07.078975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798d1dc-d468-460c-8029-edc536a4e381 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.122742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.158350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83f982-f5ef-4cd2-96aa-e25743866e36 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.208161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.209754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.209948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798d1dc-d468-460c-8029-edc536a4e381 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.210418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.210884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.212432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.214222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803c61-89c9-4be5-8009-dff632f252eb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.216185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 201/411] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:06 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.217671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.218277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.218768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.238765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:07.240943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:07.255333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.256697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.257165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.257512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] PATH_INFO: `/v3/groups/853c78aab5754b7cab42d2a242568d55` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.260054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] RBAC: Authorizing `identity:get_group(group_id=853c78aab5754b7cab42d2a242568d55)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.262981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.266587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 211/412] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/groups/853c78aab5754b7cab42d2a242568d55 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.268134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.328885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 202/413] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:07.329831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.369054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.387907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec302af7-b5a1-47fd-83fc-f44721dbf7fd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.390776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.436896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.438013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.438595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.439071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.443194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.445031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.447183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 212/414] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.452749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.491727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3dcaa-16be-4ad8-8390-b3b16500d3a5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.566267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fd86a-0257-45c0-b2da-e6396077544e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.586219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.586857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.587132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.587400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] PATH_INFO: `/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.593346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=8d87d471ae2b4849a0dda41c93ab2dfd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.594784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.596674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 203/415] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd => generated 344 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:07.602483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.605616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.606593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.607132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.607602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27/groups/89271033f16444eb9fcc320c5df6a3b8/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.610105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c677764ecb74305a5ec39c26cce1a27, group_id=89271033f16444eb9fcc320c5df6a3b8, role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.611944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.709570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.722985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1165-f80c-470d-b1ed-44cda36b3ee7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.758952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.759552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.759792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.760040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] PATH_INFO: `/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd/groups/853c78aab5754b7cab42d2a242568d55/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:07.762154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d87d471ae2b4849a0dda41c93ab2dfd, group_id=853c78aab5754b7cab42d2a242568d55, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:07.763460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:07.770444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 213/416] 10.222.0.72 () {58 vars in 1442 bytes} [Wed Aug 24 07:36:07 2022] PUT /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27/groups/89271033f16444eb9fcc320c5df6a3b8/roles/0bb62a6e624c40b4832f54f88761160b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:36:07.772470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:07.814037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:07.846066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 204/417] 10.222.0.72 () {58 vars in 1442 bytes} [Wed Aug 24 07:36:07 2022] PUT /identity/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd/groups/853c78aab5754b7cab42d2a242568d55/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:07.926712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054aa6c-c796-401c-9911-d30199608611 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:07.981070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:07.981790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:07.982078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:07.982351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.001635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:08.006749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:08.007686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:08.008509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 214/418] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:08.014189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.171207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6277b-67b1-46a4-9e1c-0c346e2c5b43 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.212257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.212899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.213178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.213450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.214114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] RBAC: Adding query filter params (name=member) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:08.214412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.215896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2603e67-89f1-4373-8016-91c5f4018023 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.230390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 205/419] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:08.231799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.370205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0f481-a93f-41c7-b8af-ae02b1582398 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.370547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0f481-a93f-41c7-b8af-ae02b1582398 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.370733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0f481-a93f-41c7-b8af-ae02b1582398 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.372380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 206/420] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:08.382034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.382327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.382578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.383050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.393400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36023aa8-3cbe-4a17-9388-920fa26bdc0a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.424592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:08.425827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:08.429744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.436563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.437268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.437552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.437824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.538159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 207/421] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:08.547129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.548691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.693139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:08.696990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 215/422] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:08.699324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.840998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14ce7-261a-4198-89dc-86ce3668f09b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:08.865450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.910521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:08.911156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:08.911436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:08.911765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:08.914842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:08.916290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:08.918270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 208/423] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:08.923603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:08.945908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.039394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dae6f-d481-46c6-a692-332d8cd84cea None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.065557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e7931-f646-4251-b970-d70e9b7a127a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.080433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.081125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.081402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.081671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.096638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a2a5df44-3369-47a8-88d3-bf2b51278bdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:09.101899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:09.102815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:09.104199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 216/424] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:09.104877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05f71-3b00-420b-9c87-b3618e576a4b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.104877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05f71-3b00-420b-9c87-b3618e576a4b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.105282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada05f71-3b00-420b-9c87-b3618e576a4b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.106331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 217/425] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:09.108672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05f71-3b00-420b-9c87-b3618e576a4b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.110402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.111253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.111712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.112160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.113504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.115144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f16a68-7257-4dfb-8fa0-950275d2f747 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.120773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 209/426] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:08 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:09.122273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.122730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.123163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.142861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:09.144311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:09.168975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.217156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 210/427] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:09.218375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.225865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada05f71-3b00-420b-9c87-b3618e576a4b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.227016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.272538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.273847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.274274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.274697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.275507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] RBAC: Adding query filter params (name=member) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.275969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.278374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.290076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 218/428] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:09.295692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.333063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73af829-fee2-4b7a-bb40-3cffb1dc27dd None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.368798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.369534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.370206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.370206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.371160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.371613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.373223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.384813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 211/429] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:09.409177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f95311-a4a2-45fc-b2fd-ecb972808a04 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.441063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.450040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.450897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.456331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.456331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.529547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.531251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.635019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 219/430] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:09.641094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.669699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.688070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.739702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.783476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc10b9-1055-4436-b37c-8ca4c48c073d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.791296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fae10-51f3-4407-aba5-df8cc1a5349e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.826070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.826955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.827452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.827562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.827966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.828135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.828413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.828706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.829791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.831561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.833369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 212/431] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:09.839796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.840524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:09.845404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:09.846503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:09.847481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 220/432] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:09.852264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:09.931184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:09.956449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ca9f7-eef2-4d8e-b958-f49316ca3e8e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.957788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd79f3-f6bf-49d8-b748-260238419066 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:09.993911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.994565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.994849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.995120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.995677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:09.995803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:09.996079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:09.996844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:09.997297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:09.997488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:09.997920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:09.999272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.000921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.002966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 213/433] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:10.007925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 221/434] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:10.009061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.011522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.022530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.133211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d53ea1-ba18-4e19-9e7f-761ca70fcb5e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.136900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7caeec-23da-4077-bc5d-622e8c03d0a6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.180854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.181791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.182246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.182716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.182828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.183445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.183726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.184000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.184311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=6d9e6b6149d7497d86e9d5fd0ca6876b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:10.184818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.186439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9b489-52b6-4788-aca0-f9f15f9ff0b6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.187643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.195254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 214/435] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=6d9e6b6149d7497d86e9d5fd0ca6876b => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:10.200987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e50f9-c834-4a05-b53a-0488ab4c8b89 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.201518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e50f9-c834-4a05-b53a-0488ab4c8b89 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.201797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e50f9-c834-4a05-b53a-0488ab4c8b89 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.203443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 215/436] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:10.209647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5014-ccd0-4d6c-9aee-4e1feb9fa842 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.210296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5014-ccd0-4d6c-9aee-4e1feb9fa842 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.210768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5014-ccd0-4d6c-9aee-4e1feb9fa842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.233888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ed0d5014-ccd0-4d6c-9aee-4e1feb9fa842 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:10.235254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ed0d5014-ccd0-4d6c-9aee-4e1feb9fa842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:10.272565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.274370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.322339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.336719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 216/437] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:10.336719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd68e3-8880-4dc2-a118-44b2cf1a4c86 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.336719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd68e3-8880-4dc2-a118-44b2cf1a4c86 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.336719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd68e3-8880-4dc2-a118-44b2cf1a4c86 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.336719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 217/438] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:10.340298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.340808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.343190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.365319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:10.365849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:10.424694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 222/439] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:36:10.431742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.481518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.492674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 218/440] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:10.546496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb699492-19ee-499d-a457-2cb199aaafcc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.580171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.580827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.581106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.581376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] PATH_INFO: `/v3/projects/6f3e29dc1cb740dcae1432624787cdb3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.582490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] RBAC: Authorizing `identity:get_project(project_id=6f3e29dc1cb740dcae1432624787cdb3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.583963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.585740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 223/441] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/projects/6f3e29dc1cb740dcae1432624787cdb3 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:10.591210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.656547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.695434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3362ac-d453-40aa-8fa4-0779f8d5c1cc None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.726446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.732239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.732792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.733070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.733339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.734696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.736120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.737709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 219/442] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:10.754514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.768075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99b351-ca9b-431e-9d96-b987e23e3c48 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.770393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.803331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.803960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.804499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.804805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.815193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:10.823048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:10.823999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 224/443] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:10.824823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.848358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d953f5-2376-405a-a597-468e316bd19f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.881461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.882051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.882324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.882596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.883306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=6f3e29dc1cb740dcae1432624787cdb3) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:10.883605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:10.885186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:10.890874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 220/444] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=6f3e29dc1cb740dcae1432624787cdb3&role.id=0bb62a6e624c40b4832f54f88761160b => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:10.891638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:10.928268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2974bc-8db0-46e7-bb17-ce0844d3d377 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:10.966419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:10.967355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:10.967633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:10.967904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:10.971827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:10.980891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:10.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:10.993171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:10.993968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 225/445] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:10.995717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.014499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e550694-b285-4e1f-90c3-81d4ec436938 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.024243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.057964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.058590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.058869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.059140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.059805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:11.060111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.061526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.071219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 221/446] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:11.077593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.119555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a7b7-526c-4f55-ba25-6b008ee9f730 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.159422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.160096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.160380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.160693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.162001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:11.162001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.163051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.172153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 226/447] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:11.177893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.199229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c240d01-138e-45a0-9faa-037b6f90c68f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.227426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.241376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.245776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.245776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.245776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.288190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.304806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829c4dc-243b-4ff7-b5ee-d22e64561859 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.320342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.321983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.346570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.347183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.347461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.347729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.426382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.428420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.442805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 222/448] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:11.448309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.471739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.511646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.527203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 227/449] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:36:11.532893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.546908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7263a-bff8-40f1-940c-ef84c227dc43 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.581328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.581949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.582216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.582479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] PATH_INFO: `/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.584079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] RBAC: Authorizing `identity:get_project(project_id=8d87d471ae2b4849a0dda41c93ab2dfd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.585545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.587135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 223/450] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/projects/8d87d471ae2b4849a0dda41c93ab2dfd => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.592442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.631333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0958650-6486-48f3-ab2b-04a7b4d357d0 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.666623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.667242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.667518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.667804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] PATH_INFO: `/v3/projects/4c677764ecb74305a5ec39c26cce1a27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.668942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] RBAC: Authorizing `identity:get_project(project_id=4c677764ecb74305a5ec39c26cce1a27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.670368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.672236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 228/451] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/projects/4c677764ecb74305a5ec39c26cce1a27 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.677374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.682495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.691011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93230948-ecd0-4748-b5a9-8db68fe61d4a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.725650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.726204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.726477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.726743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] PATH_INFO: `/v3/groups/853c78aab5754b7cab42d2a242568d55` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.727842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] RBAC: Authorizing `identity:get_group(group_id=853c78aab5754b7cab42d2a242568d55)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.729307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.730862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 224/452] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/groups/853c78aab5754b7cab42d2a242568d55 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.731578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.736594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.775223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e737bc-091a-4f3e-9ca9-894a9fc69689 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.809462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.810089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.810520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.810695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] PATH_INFO: `/v3/groups/89271033f16444eb9fcc320c5df6a3b8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.811722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] RBAC: Authorizing `identity:get_group(group_id=89271033f16444eb9fcc320c5df6a3b8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.813216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.814765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 229/453] 10.222.0.72 () {56 vars in 1180 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/groups/89271033f16444eb9fcc320c5df6a3b8 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.820482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:11.834131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db4b9c-5544-4691-bb5e-ea4ab5a7a3a3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.881123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.882314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.882604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.882881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.883622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] RBAC: Adding query filter params (group.id=853c78aab5754b7cab42d2a242568d55, role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=8d87d471ae2b4849a0dda41c93ab2dfd) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:11.883935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.885418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704cbce-33fa-4aaa-906d-53230d9932f1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.891307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 225/454] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/role_assignments?group.id=853c78aab5754b7cab42d2a242568d55&scope.project.id=8d87d471ae2b4849a0dda41c93ab2dfd&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:11.926318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df9519-553e-4f36-8228-152cacf160b7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:11.956119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:11.969186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:11.971204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:11.971204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:11.971204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:11.971204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] RBAC: Adding query filter params (group.id=89271033f16444eb9fcc320c5df6a3b8, role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=4c677764ecb74305a5ec39c26cce1a27) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:11.971582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:11.973013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4855-6be4-496e-8f34-b7e389fb7700 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:11.979729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 230/455] 10.222.0.72 () {56 vars in 1376 bytes} [Wed Aug 24 07:36:11 2022] GET /identity/v3/role_assignments?group.id=89271033f16444eb9fcc320c5df6a3b8&scope.project.id=4c677764ecb74305a5ec39c26cce1a27&role.id=0bb62a6e624c40b4832f54f88761160b => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:14.568946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c08436-f365-4095-aa84-d4e669e74bf6 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.569455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c08436-f365-4095-aa84-d4e669e74bf6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.569455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c08436-f365-4095-aa84-d4e669e74bf6 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.571047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 226/456] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:14.576893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc18c-aded-48e7-8494-e554f652d736 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.577298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc18c-aded-48e7-8494-e554f652d736 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.577485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5cc18c-aded-48e7-8494-e554f652d736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.605692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-dc5cc18c-aded-48e7-8494-e554f652d736 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:14.606916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dc5cc18c-aded-48e7-8494-e554f652d736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:14.666620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eef8ac-314e-4345-b626-ef727da89460 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.666895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eef8ac-314e-4345-b626-ef727da89460 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.667084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eef8ac-314e-4345-b626-ef727da89460 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.669101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 227/457] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:14.675894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.676177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.676448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.695406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 231/458] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:14.709363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:14.710569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:14.737199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:14.764533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969369b5-b725-4c3c-861f-8e3bf72d68b5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.765174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969369b5-b725-4c3c-861f-8e3bf72d68b5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.765651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969369b5-b725-4c3c-861f-8e3bf72d68b5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.768677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 232/459] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:14.773845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:14.774506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:14.774998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:14.806702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:14.809787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:14.814501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 228/460] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:14.899073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 233/461] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:15.103209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.146659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.222244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.223522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab22a6-79b4-483d-a43f-140d19d6fe89 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.260391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.261294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.261719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.262155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.273643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.275504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.278712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a992c69-1be8-422d-a752-fa236066fd07 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-a1947632-41b3-4cbd-a39e-14064d22613b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:15.292830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:15.294028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:15.294870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:15.294870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.294870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:15.294870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 229/462] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:15.295550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a5b6-0f69-4a7b-b80f-d6d64acc4cb2 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.295707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a5b6-0f69-4a7b-b80f-d6d64acc4cb2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.295996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0a5b6-0f69-4a7b-b80f-d6d64acc4cb2 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.301160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 230/463] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:15.301160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0a5b6-0f69-4a7b-b80f-d6d64acc4cb2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.311483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.324641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.325366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.325643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.325917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.336452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.337934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:15.353856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:15.354927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:15.356118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:15.356118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.356118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:15.356118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 234/464] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:15.356118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.426383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0a5b6-0f69-4a7b-b80f-d6d64acc4cb2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.460820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.461448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.461733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.462007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.463213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.467391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d79a17-4262-44f8-a337-d91077bba62b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.474590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.475833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ba9b253c-24d1-454b-b019-d68db797ebef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:15.490211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:15.491100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:15.491960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:15.491960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:15.491960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:15.491960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 231/465] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:15.492784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.493065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.493335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.507907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.508528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.508843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.509118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.509768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:15.510073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.511445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:15.511554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.512646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:15.535345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 235/466] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:15.535345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.574070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.592700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 232/467] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:15.592700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.663133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fae1d1-7d78-4631-92cf-6f372fcf17ff None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.707497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.708723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.709152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.709820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.711065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:15.711790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.714010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.723344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb387a-50e2-4fe3-9877-2359b1153503 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.732721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 236/468] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:15.733743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.755822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.770495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.771317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.771741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.772257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.774012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:15.774475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.775995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.795825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 233/469] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:15.797917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:15.847214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e73b71-1b9c-486b-9262-14889b5ac575 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.853641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:15.884733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.885347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.885627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.886084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.886740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.888304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:15.910599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f39e6-5712-4237-b0ac-23ea1fadb342 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:15.949189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:15.949720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:15.949998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:15.950266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:15.950903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:15.952344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.002563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 237/470] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:15 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:16.003608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.032225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 234/471] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:15 2022] POST /identity/v3/users => generated 246 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:16.033092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.142119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.184701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1aaff6-ee93-49a2-95be-67bead713138 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.205506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d436c55-a723-4c09-b65b-bb4feaed4235 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.210111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.234178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.234394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.234619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.234894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.235544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.237053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72524552-8fa7-482f-8f0e-bf5c041e0873 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.249315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.249859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.250509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.250794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.265544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.267100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d3cf6397-eda4-44a4-83a4-a50ca4365f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:16.282709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:16.283658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:16.284495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:16.284495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:16.284495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:16.284495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 235/472] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:16.285328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af612-3b63-4c96-9fec-27aca20249ce None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.285613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af612-3b63-4c96-9fec-27aca20249ce None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.285885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6af612-3b63-4c96-9fec-27aca20249ce None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.287983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 236/473] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:16.291934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6af612-3b63-4c96-9fec-27aca20249ce None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.346293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 238/474] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:16 2022] POST /identity/v3/users => generated 248 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:16.347975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.348492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.349096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.371947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:16.373545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:16.432007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6af612-3b63-4c96-9fec-27aca20249ce None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.466321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 239/475] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:16.477600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.478218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.478489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.478756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.479412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:16.479914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.481463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.495876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 237/476] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:16.502077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.588619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.630922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a615-8021-48e4-8af3-af2a531cd57c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.667390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.668011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.668257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.668533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.669223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.670668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:16.752367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 240/477] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:16 2022] POST /identity/v3/users => generated 249 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:16.811396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:16.831946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:16.931744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960559cb-2fd9-4abf-a48b-91da8a29825a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:16.978768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:16.979770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:16.980261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:16.980752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:16.981634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:16.983316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.031395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.sql.core [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:36:17.031395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:36:17.031395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [parameters: {'id': '774589e681a849be8cc29bb8cf78091a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:36:17.031395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application result.read() Aug 24 07:36:17.048265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:36:17.054525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application yield resource Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise self.value Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application session.commit() Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:36:17.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:36:17.056287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:36:17.057207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application result.read() Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:36:17.059355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [parameters: {'id': '774589e681a849be8cc29bb8cf78091a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:17.061534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 07:36:17.064471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:17.074218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 238/478] 10.222.0.72 () {60 vars in 1131 bytes} [Wed Aug 24 07:36:16 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:17.078644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.196186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b180-e7b1-4632-bb97-ec4dfce6272f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.213795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.232932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.233979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.234460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.234952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.248152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-16e9857d-abbf-4372-8b88-03180050022a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:17.253392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:17.254268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 07:36:17.255111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:17.257758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 241/479] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:17.263733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:17.352631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:17.375031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f979d-d778-42d0-b099-09e98b3044e5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:17.417755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:17.418390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:17.418670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:17.419017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:17.419730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:17.420000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:17.421518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90f7fe-a2b8-41c6-a0ab-9d2dca7252f6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:17.432123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 239/480] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:18.321892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef460-b9ce-40a2-9bbb-dc45a9210f14 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.322408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef460-b9ce-40a2-9bbb-dc45a9210f14 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.322408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ef460-b9ce-40a2-9bbb-dc45a9210f14 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.324124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 242/481] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:18.334021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033c18-8772-4362-9bc3-6e3228d01125 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.334452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033c18-8772-4362-9bc3-6e3228d01125 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.334624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033c18-8772-4362-9bc3-6e3228d01125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.358588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-45033c18-8772-4362-9bc3-6e3228d01125 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:18.359842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-45033c18-8772-4362-9bc3-6e3228d01125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:18.413323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9a8c-1c21-4791-8340-f8bd4ce7c1fa None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.413596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9a8c-1c21-4791-8340-f8bd4ce7c1fa None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.413860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed9a8c-1c21-4791-8340-f8bd4ce7c1fa None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.415438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 243/482] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:18.420612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e705-860b-4861-9eb2-4bd20cbeffc6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.420925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e705-860b-4861-9eb2-4bd20cbeffc6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.421199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9e705-860b-4861-9eb2-4bd20cbeffc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.452843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4dc9e705-860b-4861-9eb2-4bd20cbeffc6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:18.455389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4dc9e705-860b-4861-9eb2-4bd20cbeffc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:18.472758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 240/483] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:18.556181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 244/484] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:18.654307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f10b8-f5fa-4b67-b387-bf570bcf0ed1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.654597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f10b8-f5fa-4b67-b387-bf570bcf0ed1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.654749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f10b8-f5fa-4b67-b387-bf570bcf0ed1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.656471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 241/485] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:18.662241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.662509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.662780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:18.687809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:18.689151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:18.747616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f10b8-f5fa-4b67-b387-bf570bcf0ed1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:18.779513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 245/486] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:18.890063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:18.903480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:18.936857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f10b8-f5fa-4b67-b387-bf570bcf0ed1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:18.981518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:18.984191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:18.984861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:18.985141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:18.985410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.012181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:19.018970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.019854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.021227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:19.026295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 242/487] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:19.030804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.107652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f29b00-85cf-46b9-b5df-fc904f5692b9 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.146974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.148055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.148531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.149054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.160763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:19.166208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.167090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:19.169981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788504a-6801-4bd7-aab8-7070fda605f3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.170885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 246/488] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:19.171991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.204157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.209192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.209802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.210079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.210346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.211012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:19.211312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.212810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.225570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 243/489] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:19.225570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.289374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.295301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d9c56-de10-44b9-8d60-1cd1db5ccf78 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.333946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.334604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.334883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.335159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.346156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:19.351426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.352314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:19.353137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 247/490] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:19.353800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.364448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672773d-a2a9-4877-bda5-726a3404f2ed None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.401396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.402816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.403291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.403755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.404611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:19.405145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.406798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.416080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 244/491] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:19.417146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.473180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4029c35-35a6-4669-b1a2-794c1c1e7d0d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.484197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.508752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.509186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.509463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.509737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.521803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.523353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.534848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b599d0f-6aa7-4745-b631-2fd58b408810 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:19.538307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.539213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:19.540031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:19.540031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.540031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:19.540031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 248/492] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:19.545196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.552454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.581073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.581864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.582243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.583190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.583947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:19.584357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.586068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.595987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 245/493] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:19.597101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.651322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784f5c3-db4f-4b06-89cd-84908d436c61 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.685794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.686248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.686521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.697649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.699120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.706176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54afdd-c119-4881-ad48-bcc247b3fe35 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:19.714278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.715166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:19.716043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:19.716043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.716043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:19.720245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 249/494] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:19.720993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.726342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.744414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.745058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.745336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.745607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.746302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:19.746810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.748081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262368c-e232-4de8-b824-a211d7a25bc2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.760721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 246/495] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:19.766668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a20f6-5c0c-41fc-8333-5005c28b60d7 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.767368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a20f6-5c0c-41fc-8333-5005c28b60d7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.768387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a20f6-5c0c-41fc-8333-5005c28b60d7 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.772822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 247/496] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:19.772822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a20f6-5c0c-41fc-8333-5005c28b60d7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.788256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:19.839418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cd3a6-ee1b-4cf4-be0a-01918e229c7b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.876572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.877248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.877525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.877797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.889991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.890979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a20f6-5c0c-41fc-8333-5005c28b60d7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:19.891463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:19.906872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:19.907952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:19.908909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:19.908909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:19.908909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:19.908909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 250/497] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:19.908909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:19.934495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:19.935106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.935382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.935657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.936298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:19.936602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:19.938024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc19e6-3eb1-4b12-a7d0-6320e1557c4b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:19.950625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 248/498] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:19.956016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:19.956305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:19.956674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:19.980816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:19.982224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:20.019543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.032237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9965e7-c8a3-4a46-81c1-51917b8498f1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.045765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.061101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 249/499] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:20.062063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.071466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.072210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.072492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.072817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.080801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.082369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:20.096322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:20.097257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:20.098231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:20.102769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 251/500] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:20.104182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.194565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb6d2a-9db4-4c2d-bc2d-5f15608b7ae0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.211127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7ccc4-cccc-4abf-bf5e-be820e3fcc51 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.230007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.230607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.230935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.231155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.231824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.232123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.233534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.245363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 250/501] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.246145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.251063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.251698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.251983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.252254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.261641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.263109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-fdaaa240-52cc-4663-9b19-3ae6c4cda1b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:20.274871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:20.275750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:20.276638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 252/502] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:20.277271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7c17-fb49-4c4c-ba9c-2df3fdbb783b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.277814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7c17-fb49-4c4c-ba9c-2df3fdbb783b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.277814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7c17-fb49-4c4c-ba9c-2df3fdbb783b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.282965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 253/503] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:20.282965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7c17-fb49-4c4c-ba9c-2df3fdbb783b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.283180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.315231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.357675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a2617-f72e-4aea-a617-0905459f10da None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.388993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7c17-fb49-4c4c-ba9c-2df3fdbb783b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.393202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.393809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.394084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.394353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.395651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.396225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.397420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.423107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.423731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.424007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.424276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.431529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.433017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.433606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 251/504] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.434488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-7a6bde82-a07e-446d-8e9b-a72aa147dbde None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:20.449025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:20.450028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:20.450947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 254/505] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:20.451521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.451521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.451861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.469773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:20.470995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:20.525418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.537294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 255/506] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:20.538125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.540444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca4eba-64e3-4f9e-962b-47b369114462 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.563752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.581744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.582573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.583001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.583420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.584331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.584905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.586590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.605343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 252/507] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.606469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.645986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72866f5b-7e44-4f7a-8bed-b7e217131f12 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.683232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.683861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.684126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.684398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.695168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:20.699875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:20.700779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:20.701588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 256/508] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:20.702879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.714763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83f801-7f43-442f-88ae-f59746d490f8 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.750451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.751057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.751339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.751610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.766596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.787661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:20.811209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436be9d1-aa46-4adb-9739-fe8d2b4b5cbc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:20.824575 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.826063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.846786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:20.847391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:20.847671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:20.847941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:20.848726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:20.849036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:20.850464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:20.895521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 257/509] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:20.896687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.929096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 253/510] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:20.929935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:20.972876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.003125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ffe-37f7-4657-90ee-0e6db3829c2e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.028568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f10d-d92e-40ab-930f-1c93fe9d1afb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.037638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.038377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.038676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.039014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.049221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.062066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.062661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.062937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.063209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.063871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:21.064170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.065556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.073236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 254/511] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:21.074021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.107351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.108862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.174323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5b183-4ec3-482a-9a05-4c2c9987d77f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.182348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.208564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.209210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.209487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.209754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.216983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 258/512] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:21.217907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.225182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.226672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:21.241025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:21.242072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:21.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:21.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:21.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 255/513] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:21.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.264185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.322025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a508-3c9b-4e91-9dce-ddd5a2db548a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.343671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75b59b-2376-46f0-84a4-cb432c0b1c9d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.356350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.357008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.357259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.357522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.379946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.380856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.381141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.381421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.391891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.393379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:21.409990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:21.411539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:21.412365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:21.412365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.412365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:21.412365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 256/514] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:21.412570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.429195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.429343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.430689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.472226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.520911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a2b50-d1f3-46e8-9dd6-7bc730c3671a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.534877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 259/515] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:21.536370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.554590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.555215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.555491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.555766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.567175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.568640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:21.583078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:21.584119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:21.585136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:21.585136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.585136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:21.585136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 257/516] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:21.585136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.642345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b2f9-df6b-42e7-acf3-b347589f5212 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.644412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.675939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.676584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.676891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.677167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.682857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1030e02-cb70-469a-9af9-90eaedc18746 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.685807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.687870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.689373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:21.705349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:21.706420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:21.708086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:21.708086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.708086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:21.708086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 260/517] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:21.708341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.719133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.719750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.720371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.720371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.720960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:21.721263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.722626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.734579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 258/518] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:21.735368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.818214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a5027-1bfc-4d96-b7eb-1c5dc51bf7e9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.839724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669a559-b7c0-4886-bd01-59aa183668ed None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:21.852906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.853514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.853791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.854062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.854743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:21.855049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.856427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.868019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.868895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 261/519] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:21.873574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.879125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:21.879776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:21.880052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:21.882617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:21.890199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:21.891641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:21.907119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:21.908204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:21.909258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:21.909258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:21.909258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:21.913986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 259/520] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:21.914745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:21.928840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:21.974454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d309f7-9035-4fb8-896e-ae36524398ae None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.009009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.009616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.009898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.010173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.011028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.011139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.012472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.019091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0751b-b858-4a77-98bd-919642a81178 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.025144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 262/521] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.025939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.063034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.063636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.063910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.064174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.064837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.065137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.066495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.078499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 260/522] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.079276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.113102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.128108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50684bee-d03d-41a9-9288-8eaa06de0341 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.141836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.162595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.163198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.163472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.163746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.164373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.165846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.178896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011cd7-082b-4527-bb30-5ec610961852 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.212454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.213102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.213422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.213648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.220060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.223664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:22.233808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:22.234763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 261/523] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:22.235791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.247688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 263/524] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:22 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:22.248490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.348093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.357093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783c7c7-2dfb-46e1-b8ec-7e192252fdc8 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.359962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2423c3-e8c5-4d5f-90f9-1a40afad6671 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.398876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.399563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.399939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.400304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.401093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.401396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.401573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.402033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.402312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.402585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.403382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.411920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.413434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.417113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 262/525] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.417943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:22.425645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:22.426793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 264/526] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:22.427957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.466957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.536811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123cc5-3054-4b78-9d2a-a730845ea3b2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.544803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8701427-5b1e-4597-b3af-1193d7094946 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.579464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.580072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.580407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.580527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.580758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.581175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.581454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.581565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.581725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.581815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.583754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.590227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.591681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.606668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 265/527] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.606668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:22.606896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:22.607667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 263/528] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:22.608471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.626849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.707325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.717601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58cbc8-d564-4f8c-9e49-3383e53b98ca None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.719805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4afa3-9ce4-4d7a-a74c-851b98889e61 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.756784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.757033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.757424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.757704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.757979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.758276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.758709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.759132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.760275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.761160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.762790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.764992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.766430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:22.778953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:22.779824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:22.780969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 266/529] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:22.781449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.794250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 264/530] 10.222.0.72 () {56 vars in 1139 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.794250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.876931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.902195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167278c4-2899-4d88-870b-36bc7bfd324a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.908813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45df07-8fe4-4620-8951-0a0cf85ad3a3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:22.937794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.938397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.938677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.938951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.939682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=b95e296a3e2f48599b916a0ac7be2213) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.939991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.941469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.946658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:22.947266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:22.956877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:22.956877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:22.957172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 267/531] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/role_assignments?user.id=b95e296a3e2f48599b916a0ac7be2213&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.957413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:22.957505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:22.957586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:22.957586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:22.973522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:22.982688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 265/532] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:22.983768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.085149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbdd18-c70b-48ec-a85a-8902d671c993 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.125243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7af55b-8eb1-4e5e-b4fb-03d7edb1c685 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.134024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.134920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.135488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.136129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.137412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:23.137931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.139596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.153584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 268/533] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:23.154818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.174810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.181899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.185044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.185044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.185044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.237149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.270691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818c24a-3f83-463d-8918-d022404c9377 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.271810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.273317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.307011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.307626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.307902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.308175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.308945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:23.309254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.310690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.316695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 269/534] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:23 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:23.317598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.393290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 266/535] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:23.399756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.436175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.498982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc709-d95d-4eb6-adb9-6f90bdff3d5e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.517137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1719b7-dcf3-4f6d-80da-54ee7e5f4b9f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.560028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.561138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.561425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.561702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.573342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.575283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:23.592031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:23.593078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:23.593891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:23.593891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:23.593891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:23.593891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 267/536] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:23.599944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.601359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.603696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.603977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.604249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.606049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=8a2fd7d0bbcb4a04861d7cf060c671a8) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:23.608718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.610186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.621007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.623987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 270/537] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:23 2022] GET /identity/v3/role_assignments?user.id=8a2fd7d0bbcb4a04861d7cf060c671a8&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:23.748882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:23.766726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af93fb-7ff4-4411-a74f-dde4622ece05 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:23.810058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:23.810775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:23.811115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:23.811446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:23.812164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:23.812837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:23.814289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:23.833868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 268/538] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:23.839481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:23.965514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88c88d-c968-4254-9459-8a17407fedc6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:24.009419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:24.010325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:24.010766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:24.011230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:24.019104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:24.021391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:24.035129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:24.036228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:24.037288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:24.038953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 271/539] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:24.044780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:24.202500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c035422-104d-4dc0-bbbd-b74ffb911774 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:24.248950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:24.249567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:24.249848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:24.250122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:24.250825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:24.251133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:24.252584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9621980-cc9e-49fd-b57e-4a97edc11dd9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:24.270682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 269/540] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:24.276095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:24.371179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:24.431377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e26f-3a46-4c1a-9e32-88a126f8b7fd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:24.443150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48cab-ed78-4bfc-8af7-dda447b47e52 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:24.444059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48cab-ed78-4bfc-8af7-dda447b47e52 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:24.444581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48cab-ed78-4bfc-8af7-dda447b47e52 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:24.450247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 270/541] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:24.451950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429085f-2f51-4232-b94e-177e3d25d8f8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:24.452464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429085f-2f51-4232-b94e-177e3d25d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:24.452980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429085f-2f51-4232-b94e-177e3d25d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:24.464706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:24.484235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2429085f-2f51-4232-b94e-177e3d25d8f8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:24.487424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2429085f-2f51-4232-b94e-177e3d25d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:24.495150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:24.495768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:24.496175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:24.496451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:24.497216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=c8651a702d1d4681be3ed1ac16ce9b39) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:24.500861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:24.502373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:24.516740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 272/542] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:24 2022] GET /identity/v3/role_assignments?user.id=c8651a702d1d4681be3ed1ac16ce9b39&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:24.612854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 271/543] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:25.022249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.102397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f26-8bf3-408f-b7ce-1e715d9d19f7 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.102732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f26-8bf3-408f-b7ce-1e715d9d19f7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.102944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4875f26-8bf3-408f-b7ce-1e715d9d19f7 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.104288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 272/544] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:25.109429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.109711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.109988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.129272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:25.130476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:25.163350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998877a5-908a-4b05-bf58-a9040df8bc00 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.213237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:25.233088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 273/545] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:25.242301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.242958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.243239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.243510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.258611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:25.263826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:25.264745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:25.266134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 273/546] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:25.271186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.363441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:25.407944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f20a6f-78de-4ba4-94fc-85b8cd7cd3bf None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.453630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.455118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.455393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.455664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.456325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:25.456625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:25.458059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcfa32d-a7db-4eb0-8e88-164e22a25e18 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:25.466555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 274/547] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:25.474206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.523476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4990ee-b1d0-4820-895a-1beb8e09bf36 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.524109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4990ee-b1d0-4820-895a-1beb8e09bf36 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.524599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4990ee-b1d0-4820-895a-1beb8e09bf36 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.527844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 275/548] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:25.532892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.533668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.534149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.555757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:25.557330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:25.623136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a813c3-7559-4454-9ea2-769e68d5214f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.659888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.661483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.661964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.662438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.674187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:25.676766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:25.692836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:25.693738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:25.694599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:25.694599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:25.694599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:25.697381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 274/549] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:25.703363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.722057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 276/550] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:25.722057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:25.772640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:25.839993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:25.898904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99da13-0729-4bb9-ac38-1adf448d02d0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.935253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793f4b3-765c-4dae-bc8d-d755e23904ce None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:25.964626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.965630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.966159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.966745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.982114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:25.991120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:25.991970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:25.992536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:25.993022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:25.993880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:25.994415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:25.996015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:25.997877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:25.998760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:25.999574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:26.001942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 277/551] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:26.006810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.020247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 275/552] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:26.021128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.146475 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.149318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e715749-9cdc-4182-9b83-0104788ba841 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.153196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe3a6-8a52-425d-a850-949c94796e69 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.169951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.190483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.191208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.191813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.192013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.197554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.198565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.198985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.199415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.200677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:26.201125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.202912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.203396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:26.209781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:26.210868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 278/553] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:26.216568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.222065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 276/554] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:26.223740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.328904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c6a83-467a-4646-be8e-c4ce1fb79ed0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.352862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9eab8-126a-4904-80a0-a95e085ccc5f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.365385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.366016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.381387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.381387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.381387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.381387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.391994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:26.392184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:26.393262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:26.398926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:26.399512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.399512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.399512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.399707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 279/555] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:26.399785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:26.399785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.399932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.399998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf779fe-c055-4871-ae10-23b843a6cfdf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.405980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 277/556] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:26.407351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536793c9-806c-4e81-8271-3cd5bda579e1 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.407759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536793c9-806c-4e81-8271-3cd5bda579e1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.408103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536793c9-806c-4e81-8271-3cd5bda579e1 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.413960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 278/557] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:26.414957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536793c9-806c-4e81-8271-3cd5bda579e1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.440127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.441415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.511624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ab1e1-92ed-4630-ab8f-6e71532a7ac0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.526542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536793c9-806c-4e81-8271-3cd5bda579e1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.550270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.552329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.553309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.554052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.562257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.562869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.563143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.563412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.564143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:26.564446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.566040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.567652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.569589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.583934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 279/558] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-03fb43b0-0f4a-4567-872e-3da3d45f49af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:26.584434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:26.586092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:26.587170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:26.587170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:26.587170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:26.588005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.592184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 280/559] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:26.593474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.593744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.594013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.611888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:26.613092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:26.674797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.684808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 281/560] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:26.685657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.697575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a0326-9d88-4997-9e7c-879840ef60e1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.714679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.733811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.734445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.734722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.734988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.745854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.747350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:26.762223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:26.763219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:26.764122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:26.764122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:26.764122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:26.764122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 280/561] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:26.764122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.809188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad047190-c39e-403a-a77e-5385fc82676e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.848132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.848772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.849054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.849322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.870506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3364d-3dc7-4e5c-99f4-d712c5bacc0b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:26.902522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:26.908350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:26.909059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.909340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.909615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.917953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:26.917953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.917953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d12e4b-8662-4398-a401-479d148d39b5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.921981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:26.923451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b11ad5-054e-4d05-ab95-fa18781466d5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:26.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 281/562] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:26.926313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ad4d-5a3b-4d02-a1c5-644c528cd5cc None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:26.926539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ad4d-5a3b-4d02-a1c5-644c528cd5cc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:26.926812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ad4d-5a3b-4d02-a1c5-644c528cd5cc None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:26.928415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 282/563] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:26.929007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ad4d-5a3b-4d02-a1c5-644c528cd5cc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:26.945918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.026183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 282/564] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:27.027485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.027815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.028051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.038447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ad4d-5a3b-4d02-a1c5-644c528cd5cc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.045338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:27.046528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:27.074637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.075241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.075517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.075782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.076418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:27.076738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.078105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.090027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 283/565] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:27.090725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.118289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 283/566] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:27.118964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.150281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.169341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.201808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebebc92-7c7c-4844-a2da-6a585fa7d5ec None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.223673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabaf5ce-7045-4367-89ca-6d1bfcb42d62 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.237343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.237968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.238269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.238538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.247226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.248806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.257510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.258139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.258423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.258693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:27.260687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.261621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:27.262538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 284/567] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.263128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.274899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:27.279660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.280557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:27.281410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 284/568] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.282114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.372612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3963-7cff-4aa5-80f7-50e6c5958d56 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.385109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.391473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afccb-6676-4120-951a-6f265f679b41 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.404212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.408773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.409400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.409672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.409938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.421012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.422450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.428292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.428941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.429588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.429885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:27.436883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.437971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:27.438915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:27.438915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:27.438915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:27.439085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.439164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 285/569] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.439234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.439302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:27.453086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.454027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 285/570] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.459669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.545576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a62a8-16b0-41b7-b82b-fde5a79252bd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.552499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8e526-d257-46ef-9d5e-333b2935c581 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.582397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.583000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.583267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.583530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.584249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:27.584546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.585989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.588197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.588834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.589113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.589388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.590050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:27.590351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.591714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.599549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 286/571] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:27.600334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.604355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 286/572] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:27.604355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.625906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.633532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.701716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62b8a9-c257-45e9-9529-a532c10ed49c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.702794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db954810-0650-4950-9952-7ef1aaa25501 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.734978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.735533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.735810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.736079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.737099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.737696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.737970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.738796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.739433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:27.739730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.741112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.746868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:27.752029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.752997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:27.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:27.754492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 287/573] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:27.754492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.754618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 287/574] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.755246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.833845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.846469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:27.852836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a16074-64c3-4a15-bbfe-593f2d6e6f54 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.856400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c92f9c-a39c-4a8a-886c-5a493164f80f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:27.885789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.886382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.886655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.886970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.887635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:27.887943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.889382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.889933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:27.910670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:27.910670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:27.910670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:27.910670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:27.910670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:27.911040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 288/575] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:27.911040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:27.917197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:27.924691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:27.926613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:27.926613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:27.926613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:27.926613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 288/576] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:27.926613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.006077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc5cd-b882-4b26-b4a7-0159d0adc727 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.021113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d8ad2-a62c-421a-bb16-2021eca74ea5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.041317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.041937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.042210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.042472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.048852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.054022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.055455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.058739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.059376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.059653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.059924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.067361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.068949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:28.070765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.071753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:28.072624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:28.072624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.072624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:28.072624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 289/577] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.072624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:28.084853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.085964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 289/578] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.087232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.090060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.173429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235002c2-a33e-458c-87f1-1fb9ec24565b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.188369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c1b1-977f-4dfa-9cae-6b6d6ef37eec None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.206275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.207395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.207675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.207948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.208613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.208960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.210320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.218074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 290/579] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.218856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.221016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.221670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.221950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.222220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.233536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.234988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:28.248616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.249583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:28.250536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:28.250536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.250536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:28.250536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 290/580] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.250536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.277247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.305294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.325248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42daa148-4026-492f-81a8-1ed203aaac16 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.349353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1924-4e9b-4c2d-8f5e-b84f0dfbf2ce None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.359265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.359883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.360158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.360428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.361098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.361402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.362759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.374282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 291/581] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.375070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.387796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.388783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.389062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.389330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.390285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.390588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.391933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.404134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 291/582] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.405109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.479496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab162c-e674-4611-91f9-333068d6ffa4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.492298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.505133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04c1f5-ddb3-4263-88ee-a4863d1fe28a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.513088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.516884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.520337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.528840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 292/583] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.529572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.541033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.541759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.542041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.542314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.552120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.553841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:28.568489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.569453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:28.570306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:28.570306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:28.570306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:28.570306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 292/584] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.570531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.634076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577410d-70a9-4f14-adf1-f66d0062e18e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.668062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.668716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.668995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.669265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.669715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653d8ca-6016-4ded-a699-a735206643c6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.669903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.670202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.671583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.683847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 293/585] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.684633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.704474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.705138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.705412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.705675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.713194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.714624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.715131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:28.726003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.734945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 293/586] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.735872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.756646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.783440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b296ca1-70a6-47ea-9ad1-c679e42bd6aa None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.818371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.818998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.819271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.819538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.826038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.827480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.829226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf0527-e235-43bb-9ee0-e9b561e589d6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:28.839608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:28.840540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:28.841454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 294/587] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:28.841933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.865848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:28.866442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:28.866717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:28.866988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:28.867705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=e977eb2e34a945b4adeffbc92d154248) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:28.868006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:28.869490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:28.874816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 294/588] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/role_assignments?user.id=e977eb2e34a945b4adeffbc92d154248&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:28.875592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:28.954235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:28.964072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3a5d-7227-4967-b826-160df0c6fa39 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:28.989633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.000687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d755b1-92df-43b9-a36f-f01f16c6bbd1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.001667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.002294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.002573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.002898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.003608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.003890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.005332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.017617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 295/589] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.018837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.047675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.048311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.048590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.048929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.056770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.058250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:29.070080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:29.071008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:29.071909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 295/590] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:29.072475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.133556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2537bdd-d24f-46da-90bb-eb8eabce1c6c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.174855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.176075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.176364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.176636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.177445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.177731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.179405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.188339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e17713-504e-4156-8d3f-ed7e0b1380da None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.202899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 296/591] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.202899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.216072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.223340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.224136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.224407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.224707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.231405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.231405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.231405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.240272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 296/592] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.241592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.258718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.316783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474d157-d0c5-49d5-a23a-833b191dc33f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.351611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3308-0a8f-4a6e-894c-7faf38b97002 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.355396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.356086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.356463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.356811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.365035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.366436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:29.379042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:29.384740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 297/593] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:29.385905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.399354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.400048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.400392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.400774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.401672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.402047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.403546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.419674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 297/594] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.420682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.464943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.497700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.498753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f90d3-a0cd-4efb-bde0-8693028ab6bf None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.531717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f1915-5446-4389-8422-2a213e930f18 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.534351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.535184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.535457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.535727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.546242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.547683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:29.562282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:29.563270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:29.564163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:29.564163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:29.564163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:29.564163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 298/595] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:29.564404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.570816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.571420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.571700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.571971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/b95e296a3e2f48599b916a0ac7be2213/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.588891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=b95e296a3e2f48599b916a0ac7be2213, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.590470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.648401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 298/596] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:29 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/b95e296a3e2f48599b916a0ac7be2213/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:29.649265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.679007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82bf0-add7-4960-8726-b108b977a270 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.706996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.722909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.723576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.723789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.724064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.724968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.725276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.726938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.753195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 299/597] 10.222.0.72 () {56 vars in 1139 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.753195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.799839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2dd-f66b-46d6-938c-de0c933bf205 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.830503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:29.849606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.850632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.850913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.851186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.869290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=d882cbdd492344c7a3638c65ed08b32d, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.871113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.882395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c05a-830a-4397-82f0-f7fb274165e0 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:29.918748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:29.919361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:29.919638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:29.919909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:29.920551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:29.920930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:29.922298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:29.939457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 300/598] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:29.944672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:29.957663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 299/599] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:29 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:29.959682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.013511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.081488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b4a7c-5b99-45bf-b66b-fda87251a5d8 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.091967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1726a5-f85e-4a1b-af9d-210b870b9ad5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.130280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.130499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.130929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.131202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.131469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.132242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.132759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.133220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.140887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.142645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.144346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.145838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:30.155063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:30.156004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:30.156908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:30.159424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 300/600] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:30.164606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:30.165572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:30.166476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:30.166476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:30.166476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:30.166476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 301/601] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:30.168435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.168644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.225053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.275724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.291845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3a3fb-b2d7-4768-abee-838580716856 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.302192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b9bb1-02e4-4cbe-9fc9-10ab9bed03b6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.335532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.336966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.337708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.338175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.339177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:30.340026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.342192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.344442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.345108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.345385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.345657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.346502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:30.346677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.348629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.358811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 302/602] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:30.363712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.371130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 301/603] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:30.372498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.476690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.487368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e3283-e4f1-4fc7-81f3-89469d9f2da6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.524237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.525501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.526176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.526665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/8a2fd7d0bbcb4a04861d7cf060c671a8/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.544617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=8a2fd7d0bbcb4a04861d7cf060c671a8, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.546909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64b890-7fc4-467c-a931-e999953ded33 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.566528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551e3ef-145d-4fb0-b806-9b56d7c4e371 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.604202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.605214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.605776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.606293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.614267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.616119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:30.631232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:30.632124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:30.633068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:30.635583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 303/604] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:30.640923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.655892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 302/605] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:30 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/8a2fd7d0bbcb4a04861d7cf060c671a8/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:30.673928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.805308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023120c5-8ed7-4996-a9c4-502066725e3b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:30.844860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d885-cc8a-4e37-aa36-69787af39fa1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.845192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d885-cc8a-4e37-aa36-69787af39fa1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.845417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76d885-cc8a-4e37-aa36-69787af39fa1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.848074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 303/606] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:30.849720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:30.850546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.851076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.851595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.852073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a418-5061-491b-a133-15bab60aeadd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:30.852406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a418-5061-491b-a133-15bab60aeadd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:30.852703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853a418-5061-491b-a133-15bab60aeadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:30.853165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:30.853667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:30.855385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:30.869693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 304/607] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:30.877669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:30.884048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9853a418-5061-491b-a133-15bab60aeadd None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:30.885538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9853a418-5061-491b-a133-15bab60aeadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:30.960403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:30.980730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 304/608] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:30.997373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75a853-cda7-499a-b36e-f54676037233 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:31.058301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:31.059389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:31.059863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:31.060323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/c8651a702d1d4681be3ed1ac16ce9b39/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:31.086027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=c8651a702d1d4681be3ed1ac16ce9b39, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:31.087478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:31.179968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 305/609] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:30 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/c8651a702d1d4681be3ed1ac16ce9b39/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:31.277489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853a418-5061-491b-a133-15bab60aeadd None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:31.371340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:31.423499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853a418-5061-491b-a133-15bab60aeadd None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:31.462416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:31.462978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:31.463258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:31.463528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:31.474079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:31.479005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:31.479898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:31.480778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:31.484209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 305/610] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:31.486591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:31.629665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defeccb5-0e44-4fd4-8c2f-b72e02becfb6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:31.654564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:31.666485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:31.667103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:31.668027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:31.668027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:31.668335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:31.668624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:31.670139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:31.683553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 306/611] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:31.684938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:31.810933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ccb80-1c00-40f2-afd5-3f1f347c8e6c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:31.840405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:31.853305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:31.854219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:31.854664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:31.855104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:31.869658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:31.871379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:31.888565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:31.889569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:31.890430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:31.890430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:31.890430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:31.891971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 306/612] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:31.897768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.028883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2341b-7ae0-4756-8c40-c5847b28bb7a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:32.064617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:32.065176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.065454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.065722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.066422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:32.066723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:32.068088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:32.080143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 307/613] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:32.087250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.239363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c7f69-5746-4625-b11c-444b9daf0045 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:32.294423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:32.295404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.295863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.296311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.307318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:32.309033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-10d2eb54-479f-4912-9c0b-151316d955ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:32.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:32.324860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:32.325819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:32.327632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 307/614] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:32.337831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.402636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d9174-157e-44ce-b6a7-d94a3853f677 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.403273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d9174-157e-44ce-b6a7-d94a3853f677 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.403775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d9174-157e-44ce-b6a7-d94a3853f677 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.405843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 308/615] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:32.412136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.412717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.413220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.415622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:32.443870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:32.446502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:32.511783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6f0-029e-4942-8649-014be22a9712 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:32.524692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:32.574904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 309/616] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:32.579012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:32.579742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.580022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.580293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.581060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:32.581886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:32.583338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f379e-502f-43c8-9b87-c9415d874671 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:32.608128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 308/617] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:32.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.747936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4344a-38f3-43bd-8b4c-87eec8ad8084 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:32.760586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c0ab-e298-4165-bc31-8cef6572db53 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.760886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c0ab-e298-4165-bc31-8cef6572db53 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.761158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c0ab-e298-4165-bc31-8cef6572db53 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.762760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 309/618] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:32.767799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.768086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.768363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.787331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:32.788807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:32.807903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:32.808569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:32.809216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:32.809495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:32.829606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:32.831661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:32.858519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:32.859528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:32.860581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:32.860581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:32.860581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:32.863473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 310/619] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:32.868944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.897318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 310/620] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:32.913870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:32.941504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:32.977033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.012792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef49c2-c0fc-4295-b856-cb9c03ec2af7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.052035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.053021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.053350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.053623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.054281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:33.054583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.055957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0c6f2-eb87-46ef-a129-c28f48c2bde9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.070812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 311/621] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:33.071015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159748b7-23a4-43f3-8cba-14568100d1ab None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.072095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8e79-e85c-4a52-9c11-29f94168e899 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.072378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8e79-e85c-4a52-9c11-29f94168e899 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.072664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8e79-e85c-4a52-9c11-29f94168e899 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.074018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 312/622] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:33.078651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8e79-e85c-4a52-9c11-29f94168e899 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.109959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.110613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.110887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.111153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.122986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-90a368a2-0342-48a3-a891-89da5ebf3e1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:33.128095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:33.130288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.132401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:33.140748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 311/623] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:33.140748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.140748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.140748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.155930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:33.157274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:33.167148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.184790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8e79-e85c-4a52-9c11-29f94168e899 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.219902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.220538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.221106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.221106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.229111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.230548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.236975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 312/624] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:33.236975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:33.248741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:33.249830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:33.250781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 313/625] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:33.251351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.266750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.355823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63087d16-561d-4141-a007-35ac6fe4f0a2 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.392483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7746d-5888-4629-9eb5-508bebb4e2bc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.393332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.394550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.394894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.395248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.395972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:33.397080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.397874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.416118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 313/626] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:33.416118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.447099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.448702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.449182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.449657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.463713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:33.470118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:33.471007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.471820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:33.474314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 314/627] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:33.474942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.481599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.510655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.527233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f71ab-6c41-46da-8223-fffeb3887cc2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.561386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.561988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.562263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.562532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.563227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:33.563526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.564976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.576442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 314/628] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:33.577114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.592855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a508-a955-498e-b754-f75fa1ffa154 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.627591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.628736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.629012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.629278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.639030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.640472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:33.656148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:33.657259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:33.658309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:33.658309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:33.658309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:33.658309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 315/629] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:33.658309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.679841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05f68e-e100-4a26-8cad-0cb2abe0c4fd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.713231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.713831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.714108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.714383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.715040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:33.715556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.715689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.716734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d90f1ac-6788-4ed1-a908-6c4079c5767e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.724808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 315/630] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:33.725998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e0bf9-e4fc-49ca-a15a-8763f64781bd None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.729802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e0bf9-e4fc-49ca-a15a-8763f64781bd None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.729802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e0bf9-e4fc-49ca-a15a-8763f64781bd None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.729802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 316/631] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:33.729802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e0bf9-e4fc-49ca-a15a-8763f64781bd None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.737541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.765329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcaabbf-4b2a-4af8-ba8c-e39d4c1663e1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.799527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.800173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.800451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.800750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.811294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:33.816972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:33.817932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:33.818785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 316/632] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:33.819484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.833044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e0bf9-e4fc-49ca-a15a-8763f64781bd None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.867317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.867968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.868245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.868516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/e977eb2e34a945b4adeffbc92d154248/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.883323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=e977eb2e34a945b4adeffbc92d154248, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.884833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f933b-60de-4831-becb-1302d4172eb7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.917772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e4584-31ff-4642-a44a-b93368c66ef0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:33.934127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:33.951209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:33.951800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.952075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.952343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:33.953019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:33.953321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:33.954726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:33.965758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 317/633] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:33.966611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:33.980893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 317/634] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:33 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/e977eb2e34a945b4adeffbc92d154248/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:33.982892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:33.983113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:33.983328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.006000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:34.007224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:34.063866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.084320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7903d-6912-47a5-af05-5d69099bc481 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.097546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 318/635] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:34.098508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.136126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.136799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.137082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.137354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.151489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.153025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:34.168452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:34.169602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:34.170486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:34.170486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:34.170486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:34.170486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 318/636] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:34.170713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.200839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.247924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f2e13-89be-4c2b-b36c-36ac41f5be74 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.280825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7d2-97c2-4303-a832-fd956b396ad1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.294288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.295502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.296444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.297094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.298071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:34.298839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.300946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.316639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 319/637] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:34.316639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.328640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.329397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.329679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.329950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.341847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.343452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:34.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:34.360452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 07:36:34.361749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:34.365361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 319/638] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:34.366269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.389759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.452271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c4649-2475-4565-a030-616f7d559676 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.463642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.476166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed619fc-c25d-460d-ad16-f8d30cde1a5f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.488624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.489515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.489797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.490069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.490720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:34.491027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.492547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.505439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 320/639] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:34.506258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.517164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.517823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.518095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.518366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.530314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.532018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:34.548197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:34.549384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:34.555520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:34.555520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:34.555520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:34.555520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 320/640] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:34.555520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.613044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad360-a2ff-4f9d-bb89-0eb67628ebb9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.626078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.654596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.655298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.655639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.655976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.656811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:34.657213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.658706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.667666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c68952-b3a8-487b-bf3f-3c288bc72308 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.676092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 321/641] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:34.677160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.707706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.708413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.708814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.709187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.716790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.720772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:34.725459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:34.732309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 321/642] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:34.733211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.790892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3b1dd-1578-4413-bf20-f5f0e878c7ec None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.829453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.830378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.830807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.831263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.834146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7c27e-5a5e-482c-a080-f42ed2f5c6f7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:34.838885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.840485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:34.852697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:34.853717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:34.854937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:34.857082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 322/643] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:34.858013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.874354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:34.874988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:34.875245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:34.875490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:34.876117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:34.876415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:34.877873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:34.890803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.892915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 322/644] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:34.893988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:34.958745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:34.981070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4a7fa-3a33-4d4d-9bc6-8a95a0313f2c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.006253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9000201-de5c-4bdb-90a6-5a75bfc80441 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.019797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.020457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.020994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.021140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.042077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.042680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.042958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.043228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.043892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:35.044189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.045591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.055157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 323/645] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:35.055973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.102466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.103930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.132196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.167502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd745d-b439-4bf2-966c-b3eb7783f532 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.190331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.203130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.203732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.204007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.204277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.205028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:35.205331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.206855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.212622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 323/646] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:35.212622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.234816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 324/647] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:35.234816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.323573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c36d4-0403-4fbb-b3ec-0760582f880f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.345603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785fd9-7114-4268-9d1f-4ab4c0ae2246 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.350778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.359689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.360393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.360752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.363144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.369366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.370861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:35.381828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:35.382986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:36:35.384155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:35.385098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 324/648] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:35.385208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.391970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.392605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.392925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.393195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.404313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.405892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:35.422908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:35.423932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:35.424930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:35.424930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:35.424930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:35.424930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 325/649] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:35.424930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.462588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.501561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33f192-9bb7-4991-a27f-786e24efd15f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.537421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4396-769b-4671-9a7b-d11177884e3a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.538376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.539010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.539289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.539557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.551647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.553435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:35.568022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:35.569253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:35.570582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:35.570582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:35.570582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:35.570582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 325/650] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:35.570582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.578293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.579480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.579863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.580236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.613320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.653634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.655337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.682286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24a3a-584a-4a38-ac72-02689eddce33 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.687055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.719949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.720935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.721373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.721719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.723018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:35.723323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.724895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.744057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 326/651] 10.222.0.72 () {56 vars in 1137 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:35.744057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.763046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 326/652] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:35.763877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.852436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.859516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aca6-c917-41d1-a04f-3c895fd8f02e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.874367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd58be-fee8-4264-881d-dc2272214055 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:35.897963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.898581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.898854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.899119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.899759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:35.900061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.901446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.906926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:35.911701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:35.912299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:35.912573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:35.915310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:35.915446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 327/653] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:35.915574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:35.915704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:35.915802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:35.915802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:35.926430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 327/654] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:35.927234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.015994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa3a2d-aa9b-485a-852a-fc0c03bd5bc3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.027333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece1ebd-4829-4b18-bce7-574b4339cd9a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.049337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.049965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.050242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.050508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.061905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.062529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.062806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.063081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.074245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.075003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.075695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2616e090-6fe5-49ef-bcab-0d857bbfa731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:36.090294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:36.091247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:36.092135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:36.092135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:36.092135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:36.092135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 328/655] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:36.092892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbda1-4eed-4d42-a16e-e13cca16d85e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.092892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbda1-4eed-4d42-a16e-e13cca16d85e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.093128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbda1-4eed-4d42-a16e-e13cca16d85e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.094369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 329/656] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:36.095045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bbda1-4eed-4d42-a16e-e13cca16d85e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.121734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.123188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.149993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.214240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bbda1-4eed-4d42-a16e-e13cca16d85e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.226104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 328/657] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:36.226939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.248058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.248708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.248988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.249256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.257374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.258813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:36.270353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:36.271269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:36.272216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 330/658] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:36.273378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.285855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.331538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5670a-448c-4d1b-a728-dc17b5a8c6f5 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.369351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.369960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.370229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.370495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.372736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804d292-74e0-4d7a-a71d-3ad2ceba963d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.375023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.377093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.379128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a3d87852-19ef-4013-9a24-becce559469b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:36.390614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:36.391644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:36.392556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 329/659] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:36.393092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.393171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.393606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.411433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.411843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:36.412068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.412346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.412614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.413072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:36.413323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:36.413625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.414995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.428090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 331/660] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:36.428942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.484957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 330/661] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:36.485796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.533828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.535234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174af9a-e161-4689-8c18-0c011ba76071 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.573152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.573776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.574052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.574323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.584582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.586141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.595933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcb9fa-6504-44ee-9a80-3e6b4d1aa40a None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:36.601532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:36.602644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:36.603566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:36.603566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:36.603566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:36.603566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 332/662] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:36.603797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.625237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.641734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.642367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.642796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.643199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.643993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:36.644299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.646258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.669208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 331/663] 10.222.0.72 () {56 vars in 1139 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:36.670539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.728900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea594a-fa1b-4625-8847-adc9c88bcd24 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.765695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.766303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.766579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.767055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.767536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:36.767839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.769356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.781233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 333/664] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:36.781666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f9107-3d92-4c33-99b0-e48ae29f885d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.782026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.785451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.817906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.818536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.818812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.819088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.825706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.827145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:36.838126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:36.839047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 332/665] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:36.840434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.866514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:36.889085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c420b-c0f9-418f-9a42-30d1a1d83ece None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.923225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.923825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.924101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.924373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:36.925600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:36.925899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:36.927257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:36.938392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 334/666] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:36.939169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:36.939544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d55c2-31df-454c-bbda-4bd017745b85 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:36.976380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:36.977039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:36.977317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:36.977609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.015011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.044417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307dc21-bd3d-4f7f-b07a-ca4f8b987453 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.054007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.055455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.078632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.079258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.079548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.079825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.082962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.091527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:37.096215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:37.097900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:37.098730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 335/667] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:37.099420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.156477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 333/668] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:37.157343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.197863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b41bcb-606e-41b7-a1f5-90b0e5f20165 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.222798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.232311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.232935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.233220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.233484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.234206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=b95e296a3e2f48599b916a0ac7be2213) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:37.234533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.235975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.241716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 336/669] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/role_assignments?user.id=b95e296a3e2f48599b916a0ac7be2213&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:37.242493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.283921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74bf7b-90e4-40ed-bdc2-d3d6cacc95d1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.319525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.324959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.325640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.326002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.326274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.327059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:37.327374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.329743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.349998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 334/670] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:37.349998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.389728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc210f-31af-48b5-b8c3-c1e91b58eccc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.432358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.433029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.433308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.433580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.444778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.446491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:37.459473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:37.461860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 337/671] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:37.463660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.486028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2ff10-b2d2-495d-915c-c30c29166af1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.514694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.529401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.530142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.530421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.530690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.531349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:37.531646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.533260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.544230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 335/672] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:37.545115 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.588077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194f90-20ec-47e5-8606-a7ad538ff2fb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.610777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.629474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.630152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.630436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.630704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.644286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.645779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:37.660533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:37.662052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:37.663608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:37.663608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:37.663608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:37.663608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 338/673] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:37.663608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.668092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cac77-7f20-490f-a258-994b41c07690 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.705431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.706680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.706954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.707221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.707948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:37.708248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.709721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.724936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 336/674] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:37.726555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.812437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5a33a-ff00-4353-8b25-a1ad8000f948 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.814587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.857319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.858010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.858398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.859030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.859569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:37.860520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.862128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.879624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 339/675] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:37.880564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:37.887555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcede9-9573-4f68-83f5-e46426b88643 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:37.916284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:37.932032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:37.933726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:37.933726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:37.933726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:37.948887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:37.950370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:37.968461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:37.969493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:37.970562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:37.970562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:37.970562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:37.972265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 337/676] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:37.972265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.007288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7938-84fb-43f5-bd9f-ec9df46b93f1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.058966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.059586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.059868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.060147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.060996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.061307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.062693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.079161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 340/677] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:38.079375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.117986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe94f-d198-4930-92c8-466bfd5a3b0f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.120405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.157177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.158137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.158741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.159249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.160220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=8a2fd7d0bbcb4a04861d7cf060c671a8) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.160777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.162482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.169032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 338/678] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:37 2022] GET /identity/v3/role_assignments?user.id=8a2fd7d0bbcb4a04861d7cf060c671a8&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:38.170136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.199710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.210285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc8fed3-992b-4421-96ba-b5f1ab9812f5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.249606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.250261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.250544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.250819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.251491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.251795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.253835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.269537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 341/679] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:38.276200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.295881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8fce9-c385-423d-ac6c-9fde9a39dd18 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.360689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.361850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.362370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.365153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.374979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.380931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:38.394617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:38.395567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:38.396508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:38.404719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 339/680] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:38.405868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.442566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.444043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504d1ef-f68a-45e1-9cfa-a013bca57ce3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.484441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.485106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.485570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.485852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.495240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.496760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:38.509941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:38.510974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:38.512056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 342/681] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:38.516937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.536138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.552816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44329eb-cb75-40b0-8adc-b207c0e551fa None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.616986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.617842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.618124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.618468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.619172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.619476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.643587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 340/682] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:38.651568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.679968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f525ba7-5381-44c9-a533-55507dc9a26b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.751545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.752167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.752444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.752785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.754023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.754328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.755789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.787749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 343/683] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:38.796323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:38.816248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.843355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed5ab-f276-41c8-8800-971c9c7dc3a9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:38.908901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:38.909804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:38.910312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:38.911378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:38.912114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=c8651a702d1d4681be3ed1ac16ce9b39) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:38.912718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:38.914165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:38.924439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 341/684] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/role_assignments?user.id=c8651a702d1d4681be3ed1ac16ce9b39&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:38.930559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:38.966577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852c03b-21d8-40c4-9fd8-d5cfce3c6fcd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:39.014098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:39.014715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.014993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.015263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.100798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:39.102283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:39.225108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:39.260290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 344/685] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:36:39.265987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:39.375198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10612f1-dabe-4d91-af6c-a286bf1aa802 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:39.412003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:39.412642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.412947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.413218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.424275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:39.426332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d29fabad-09ef-43af-b03c-95a30de6ddd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:39.443137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:39.444731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:39.446307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:39.446307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:39.446307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:39.446307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 342/686] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:39.449920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:39.482818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301125a-9553-46cf-949a-93e03cb66eeb None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.483113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301125a-9553-46cf-949a-93e03cb66eeb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.483340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5301125a-9553-46cf-949a-93e03cb66eeb None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.485033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 343/687] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:39.495302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.495723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.496095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.516849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:39.518201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:39.528028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:39.621660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 344/688] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:39.638532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c87de-940d-4b14-a824-0883dea918ac None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:39.675500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:39.676111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.676393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.676677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.677585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:39.677887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:39.679287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:39.693587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 345/689] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:39.701553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:39.828083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553adf8-ba5c-49e1-a072-5dff2793dddc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:39.869826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:39.878282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:39.878923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:39.879200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:39.879468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:39.886671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:39.888111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-2574dd3c-a600-4b77-bdaa-9708605995ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:39.905439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:39.906381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:39.907277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 345/690] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:39.913655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:39.955483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.039667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1c3c-a780-4bd1-966d-b03c64aaa82e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.039996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1c3c-a780-4bd1-966d-b03c64aaa82e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.040222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e1c3c-a780-4bd1-966d-b03c64aaa82e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.042154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 346/691] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:40.046942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.047464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.047836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.063785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fe012-d30b-421b-a56f-660a64696403 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.070547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:40.072447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:40.123185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.123723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.124159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.124528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.125253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:40.125559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.126977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.140118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 346/692] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:40.149443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.194281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 347/693] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:40.195142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.323305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9284d-978c-46b1-9527-73262e542e50 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.340032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb7523-35c2-4b79-a823-bd20002f3634 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.343151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.374982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.375931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.382714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.383279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.383916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.384912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=e977eb2e34a945b4adeffbc92d154248) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:40.385522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.387169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.394607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 347/694] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/role_assignments?user.id=e977eb2e34a945b4adeffbc92d154248&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:40.396338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.400845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.400845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.400845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.412678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.412678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:40.431086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:40.432097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:40.433066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:40.433066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:40.433066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:40.433066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 348/695] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:40.438316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.492327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.591066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8cab1-0b9b-4775-bf5f-923a352ffe5e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.635561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9892077-b234-4981-9141-7961dd926986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.636547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892077-b234-4981-9141-7961dd926986 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.637072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892077-b234-4981-9141-7961dd926986 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.637560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9892077-b234-4981-9141-7961dd926986 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.638571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9892077-b234-4981-9141-7961dd926986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:40.638880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9892077-b234-4981-9141-7961dd926986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.640250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9892077-b234-4981-9141-7961dd926986 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.649099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca633b0a-24a6-44d5-a417-ac06d0e0f81e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.661751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 348/696] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:40.667698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9892077-b234-4981-9141-7961dd926986 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.685264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.715497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.716571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.716870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.717136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.729242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-2205163a-a175-4aa8-a171-5ee40ea47bb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:40.735263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:40.736302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:40.737544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 349/697] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:40.738305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766721f-0325-4295-bd44-c2acd3eed8bf None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.738305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766721f-0325-4295-bd44-c2acd3eed8bf None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.738305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766721f-0325-4295-bd44-c2acd3eed8bf None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.739622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 350/698] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:40.744555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766721f-0325-4295-bd44-c2acd3eed8bf None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.788938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:40.790002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9892077-b234-4981-9141-7961dd926986 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.841960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.842187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.842286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.842968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.843193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.844644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.872545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766721f-0325-4295-bd44-c2acd3eed8bf None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:40.908522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:40.909119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.909396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.909663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.910325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:40.910622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:40.911992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41435cc-73d8-4b21-a3ac-036616fd1fc6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:40.920622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 351/699] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:40.925097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:40.925097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:40.925097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:40.939360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 349/700] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:36:40 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:36:40.941046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:40.952460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:40.953722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.032884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 352/701] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:41.046036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.085679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1522539-1693-44f7-af2c-202d09310776 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.130543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.131453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.132047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.132947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.145967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.147732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:41.163295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:41.164462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:41.165462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:41.165462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:41.165462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:41.166966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 350/702] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:41.174265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.206266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.285128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01916-f8c9-4bb6-bf81-33f5c2a4e53f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.323333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.324190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.324467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.324770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.325468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:41.325768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.327195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.339356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 353/703] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:41.344640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.366427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.479171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.496127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbba99-3b93-4329-b5d1-f93bb5f581de None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.497972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e366c0f-74a7-4b6a-b9c7-c6ebc28f83c5 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.527160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.537671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.538556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.538883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.539089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.539917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.540528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.541245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.541363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.541619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.542013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.550200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.551741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e410c8ff-0cc9-49b4-be6b-bbb2803c7cbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:41.554848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:41.555978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:41.556934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:41.557709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 354/704] 10.222.0.72 () {56 vars in 1129 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:41.560879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5050a6-b67c-4afc-85cf-f542415aa462 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.560879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5050a6-b67c-4afc-85cf-f542415aa462 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.560879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5050a6-b67c-4afc-85cf-f542415aa462 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.560879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 355/705] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:41.565144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5050a6-b67c-4afc-85cf-f542415aa462 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-f25a6e8b-a664-4508-b7e1-717bf8ff720b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:41.571792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:41.573816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:41.575445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 351/706] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:41.576685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.576685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.576685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.600697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:41.602833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:41.682495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5050a6-b67c-4afc-85cf-f542415aa462 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.692811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 352/707] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:41.693902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.720913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.722097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.722369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.722635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.723260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:41.723585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.725269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.730605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 356/708] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:41.737585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.770264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.858814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce3a73-45b6-4556-811c-d16dbd8f0eb5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:41.875468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:41.900379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:41.901250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:41.902248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:41.902688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:41.903762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:41.904243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:41.906250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:41.928777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 353/709] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:41.934039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:41.942884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc011-380d-49c9-87ba-4106512a2e40 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.010688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70350-661b-4df0-9067-81795009415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.012348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70350-661b-4df0-9067-81795009415f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.012968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70350-661b-4df0-9067-81795009415f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.014460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f70350-661b-4df0-9067-81795009415f None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.015497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70350-661b-4df0-9067-81795009415f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:42.016615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70350-661b-4df0-9067-81795009415f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.018958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f70350-661b-4df0-9067-81795009415f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.031703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 357/710] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:42.046454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70350-661b-4df0-9067-81795009415f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.099166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1c8f-9d9a-4510-9dfe-cec4776c98c6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.148493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.149419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.149871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.150286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.224957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.249355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.256405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.258089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.275703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f70350-661b-4df0-9067-81795009415f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.320105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.320782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.321095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.321404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] PATH_INFO: `/v3/services/network` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.322008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.323890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:42.336238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:42.337309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:42.338763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 358/711] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/services/network => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:42.342924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.395695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 354/712] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:42.400802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.462181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7459c66-2a18-4485-aa84-cd6d318defd4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.503984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.504948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.505890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.506484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.507922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] RBAC: Adding query filter params (name=network) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:42.508746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.511155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.520495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 359/713] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:42.529649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.542773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.566697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.610559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb338ae7-b2b0-4ffc-b885-2a8e973b11e3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.648441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.649116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.649394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.649664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.665803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.667205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:42.694997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:42.696590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:42.697718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:42.697718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:42.697718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:42.708876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 355/714] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:42.709799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:42.722134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761c95e-66de-4136-b13d-72e8cc7bc859 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.760035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.760668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.760948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.761215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.763293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:42.763600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.765015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.774328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 360/715] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:42.884897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1042f8-c13d-42b3-a335-429fe5de8e9b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:42.922829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:42.938374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:42.939011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:42.939294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:42.939567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:42.940827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:42.941137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:42.942629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014027e4-a7ff-48dc-963b-2917b5c20028 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:42.965008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 356/716] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:42.967201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:43.057645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7ffc-8712-44a7-abff-8bf0f2fbe283 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.057963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7ffc-8712-44a7-abff-8bf0f2fbe283 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.058197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed7ffc-8712-44a7-abff-8bf0f2fbe283 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.059528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 357/717] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:43.066334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.066619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.066897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.070356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:43.085171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:43.086466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:43.124580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3351c60-8d26-493f-909b-b6853f18e8f2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:43.196398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:43.197580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.198734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.199360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.212419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:43.214849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:43.220451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 358/718] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:43.231572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:43.232481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:43.233415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:43.235716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 361/719] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:43.242085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:43.364071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa78dd7-cd4c-4e88-b188-e99145309e5e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:43.389115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:43.412719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:43.413392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.413677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.413952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.414656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:43.414964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:43.416444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615039-e55d-4f08-a302-8f1a7bdd5513 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:43.437943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 359/720] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:43.443541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:43.465550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f25a-d2bf-4275-a055-9be8f5a3cdd0 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.465826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f25a-d2bf-4275-a055-9be8f5a3cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.466032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068f25a-d2bf-4275-a055-9be8f5a3cdd0 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.467618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 360/721] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:43.473320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.473597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.473866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.501635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:43.503214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:43.573499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbf597-7d08-4fda-bdeb-0cd128677cf2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:43.610942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:43.614557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 361/722] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 07:36:43.618205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:43.630329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.630329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.630329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.630329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:43.630329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:43.630724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:43.630813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:43.630813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 362/723] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:43 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:43.801054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9627e-9038-48db-8034-ea0256101b79 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:43.803088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:43.849358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:43.849994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:43.850267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:43.850532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:43.870840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:43.875736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:43.876683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:43.877569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 362/724] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:43.884060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:43.913803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.062590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9b90a-4763-46dd-96b5-0177a4caaebc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.064877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e2a53-0d13-4d8f-af33-2d00fd61d6ef None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.100171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.101175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.101660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.102158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.103048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:44.103570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.105174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.114815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 363/725] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:44.120537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.139218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.139858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.140133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.140400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.163538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:44.168912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:44.169948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:44.170793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:44.177334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 363/726] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:44.187127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.214387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.240580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.273183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d7dd-a799-432b-b738-365260e9f4b2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.318882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.319434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.319711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.319980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.340684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.340684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:44.366488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:44.367802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:44.368964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:44.368964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:44.368964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:44.371332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 364/727] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:44.382514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.392427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb8e45-aefc-4a6a-97ac-cf839cab0e2b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.432185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.432836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.433271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.433545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.434687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:44.435115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.436698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d65d62-0047-4098-9162-f568b1ffb821 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.447621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 364/728] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:44.449001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c0b8-c1e2-4648-8155-fc6510e951ee None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.449286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c0b8-c1e2-4648-8155-fc6510e951ee None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.449559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26c0b8-c1e2-4648-8155-fc6510e951ee None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.451330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 365/729] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:44.453525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26c0b8-c1e2-4648-8155-fc6510e951ee None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.527601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7bcf7-e77e-4e1d-945a-78445fb9c01b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.538552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.547561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.565384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26c0b8-c1e2-4648-8155-fc6510e951ee None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.565832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.566536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.566733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.567089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.567747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:44.568050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.569522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc039bd7-d13b-4fc4-a684-c30130deaf0d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.588702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 365/730] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:44.590242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.590529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.597628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.603452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.604274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.604750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.605224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.610803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:44.612084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:44.616540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.618167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:44.633289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:44.634446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:44.635513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:44.635513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:44.635513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:44.637182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 366/731] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:44.638913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.687481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 366/732] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:44.693501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.746779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c68474-d03d-4af8-82ea-3c6f266aaf0a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.785206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.786106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.786609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.787164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.790405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.796174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.797833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:44.810264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:44.811278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:44.812197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:44.814113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 367/733] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:44.820030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:44.918132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:44.927202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b146a2-f815-452b-b08e-ce7c86f43c56 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.931856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92332d-d9ef-4fa2-a1e8-f17d53d838d6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:44.967934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:44.968928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:44.969408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:44.969896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:44.970821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:44.971878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:44.973530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:44.993129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 368/734] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:44.999906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.012230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.013288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.013768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.014254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.017255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:45.017870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.019445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.042754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 367/735] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:45.046911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.098687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:45.153802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0266c1b-0ce4-438f-ac5b-0322eb0ea148 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.161273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263b24-5edf-4255-9310-3afb9faeb320 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.192690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.193336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.193564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.193825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.198568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.199170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.199442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.199710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.200347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.201729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.264463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.265932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb5724-dc21-41ce-82b0-440a59e768bd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.295402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 368/736] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:36:45 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:45.296806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.325931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:45.332281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:45.390991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 369/737] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 07:36:45.390991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d3a9-8ff5-40aa-bc38-cff435dff1c4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.390991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d3a9-8ff5-40aa-bc38-cff435dff1c4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.390991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d3a9-8ff5-40aa-bc38-cff435dff1c4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.390991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 370/738] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:45.395027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5d3a9-8ff5-40aa-bc38-cff435dff1c4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.478960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6f2d-a250-4f12-a25a-fa44977bcfe5 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.522688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.523330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.523602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.523879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.524027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5d3a9-8ff5-40aa-bc38-cff435dff1c4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.534941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.538034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e7cd753e-b213-4830-939c-c424c4402975 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:45.555382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:45.556568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:45.557602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 369/739] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:45.558642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.558925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.559198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.562114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.562842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.563234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.563594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.574789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.576396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.580977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:45.582179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:45.595471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:45.598003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:45.600293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:45.600293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:45.600293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:45.604681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 371/740] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:45.604681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.616389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:45.667333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 370/741] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:45.669337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.678972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:45.762831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bb0cf-6c18-4341-a364-178c3fe4cb3e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.828544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51855b-a095-4e95-9983-38440310652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.829183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51855b-a095-4e95-9983-38440310652c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.829461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51855b-a095-4e95-9983-38440310652c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.829727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51855b-a095-4e95-9983-38440310652c None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.831086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51855b-a095-4e95-9983-38440310652c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:45.831385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51855b-a095-4e95-9983-38440310652c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.832838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51855b-a095-4e95-9983-38440310652c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.866109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9f66-abbd-49d0-83da-de8693b97585 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:45.872424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 372/742] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:45.878603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51855b-a095-4e95-9983-38440310652c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.937936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:45.938474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.938749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.939016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.940205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:45.940547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:45.941965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8244e-cdbc-4517-b023-9c43314021f4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:45.969014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 371/743] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:45.971196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3604b-129d-4b03-9193-2373961066dd None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:45.971489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3604b-129d-4b03-9193-2373961066dd None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:45.971767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3604b-129d-4b03-9193-2373961066dd None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:45.973713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 372/744] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:45.975249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3604b-129d-4b03-9193-2373961066dd None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:45.997970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.038952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51855b-a095-4e95-9983-38440310652c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.052840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.077269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.077875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.078155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.078429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.095746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3604b-129d-4b03-9193-2373961066dd None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.131980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.132611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.132918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.133185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.139985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.141472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.148904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.150350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-4b0887dc-d8fd-41e3-bd09-56061e0885c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:46.153543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:46.154414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 373/745] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.155736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.156354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.173288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:46.175010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:46.226385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.256963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 374/746] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 07:36:46.258019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.273492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 373/747] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 07:36:46.275104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.332723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.395828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89a37b-8609-4ce8-89e1-1f8c350fc3a8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.404974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a462dc3-6b01-4e48-99b9-d5804aa422fb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.442047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.442848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.443141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.443529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.444184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.446165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.451886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.452662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.453055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.453424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.454215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:46.454621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.456245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.468593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 374/748] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:46.469397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.503278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.533356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 375/749] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:36:46 2022] POST /identity/v3/services => generated 240 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:46.534535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.594438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456fed7-c25a-4ec2-9ffb-a29279d7d0a2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.647311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.648638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.648638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.648853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.661468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.663330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:46.680224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:46.681346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:46.682437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:46.682437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:46.682437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:46.682799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53874f-3081-4d8f-9a30-1be53ad21d4b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.687485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 375/750] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:46.688224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.699577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.727831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.728575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.728972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.729497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.730672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=e977eb2e34a945b4adeffbc92d154248) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:46.731281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.733499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.741170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 376/751] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/role_assignments?user.id=e977eb2e34a945b4adeffbc92d154248&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:46.741170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.782725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:46.852820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d000e-cb0e-4502-8699-6c8a6d606826 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.925490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.926508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.927209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.927628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.930885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd21dded-7672-4c0f-aa31-b0432ffcd3bf None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:46.945806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:46.951844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:46.953051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:46.954048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:46.956330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 376/752] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:46.965658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:46.965953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:46.966583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:46.966835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:46.967106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:46.967750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:46.968050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:46.969461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:46.982269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 377/753] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:46.993916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.070518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.161105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347fff7-64ae-4894-8a1a-b0ba177836f9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.161603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0348e5-3740-4719-8e47-4b8eccb95621 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.196142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.201782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.202390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.202667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 377/754] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:47.222683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.223125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.223473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.232257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.232541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.249197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.250660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:47.273185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:47.274165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:47.275093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 378/755] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:47.281631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.345171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b7df4-7f47-4bfd-b031-a0f718ee5b26 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.399301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15972511-249b-4242-a048-37291d547f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.402516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972511-249b-4242-a048-37291d547f6b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.402516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972511-249b-4242-a048-37291d547f6b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.402516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15972511-249b-4242-a048-37291d547f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.415769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15972511-249b-4242-a048-37291d547f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.417267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15972511-249b-4242-a048-37291d547f6b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-15972511-249b-4242-a048-37291d547f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:47.433968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:47.434952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:47.435914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:47.435914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:47.435914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:47.435914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 378/756] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:47.440680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c64e-3b0a-49cc-9559-c971ee8ce7fe None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.440680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c64e-3b0a-49cc-9559-c971ee8ce7fe None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.440680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410c64e-3b0a-49cc-9559-c971ee8ce7fe None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.443202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.444020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 379/757] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:47.444950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410c64e-3b0a-49cc-9559-c971ee8ce7fe None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.446212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fd687-f002-46cc-b85b-65979604b5ef None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.480772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.481378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.481653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.481924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.482611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:47.482913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.484332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6aa786-b6fe-4dd9-9cc8-61336e933fec None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.497198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 379/758] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:47.498529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.498818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.499095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.504028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.518447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:47.519679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:47.550982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410c64e-3b0a-49cc-9559-c971ee8ce7fe None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.586107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.586721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.586995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.587265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.587984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:47.588288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.589794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.592224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 380/759] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:47.593288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.615068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 380/760] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:47.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:47.698244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4327e-aebf-4a50-a2c1-6b720d0e2079 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.729039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.737444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.738100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.738472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.738834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.739660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=0c877a807efb45a8b6475e0c8db99e98) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:47.740067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.741654 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.747820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 381/761] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/role_assignments?user.id=0c877a807efb45a8b6475e0c8db99e98&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:47.782165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2152-5775-4268-8b3d-a5a937d96d90 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:47.790284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:47.820177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:47.821089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:47.821368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:47.821664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] PATH_INFO: `/v3/users/nova` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:47.829436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:47.830913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:47.843505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:47.844516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:47.845863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 381/762] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:47.850371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.001037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.039364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea2f9-e044-40d1-9174-d2d61ca53196 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.077606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.078664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.079143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.079657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.080639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:48.081165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.083046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.102497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 382/763] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:48.107476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.133018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.190327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576103d1-1d9d-4e65-854d-bdf2a5aafabe None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.244238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.244905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.245182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.245453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.246041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.247415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:48.260018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:48.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:48.261881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 382/764] 10.222.0.72 () {56 vars in 1129 bytes} [Wed Aug 24 07:36:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:48.269049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.336387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.359632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002ade-7a77-4ece-a51e-066156368222 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.407930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.409435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.409916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.410384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.421961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.422716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606dc9-621d-4604-b186-8b947a52f541 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.424440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:48.440276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:48.441334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:48.442245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:48.442245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:48.442245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:48.444412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 383/765] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:48.450335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.466365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.466365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.466365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.466365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.466761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:48.466761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.466761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.471553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 383/766] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:48.476899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.587282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb9856-4526-4d02-9af7-4165043c7ff3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.605012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.632226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.634287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.634550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.634861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.635628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:48.635930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.637378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.644056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.655930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 384/767] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:48.663875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.724324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca871dbc-0757-40dd-b7a4-8bd98281512e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.768114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.768762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.769142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.769419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.770166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:48.770470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.771946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bde54e-1a64-4b19-a3bc-6e62cf1d0741 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.785371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 384/768] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:48.786682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83eba3-9d99-418c-93cc-fcd189eac053 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.787097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83eba3-9d99-418c-93cc-fcd189eac053 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.787369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83eba3-9d99-418c-93cc-fcd189eac053 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.788874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 385/769] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:48.793872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83eba3-9d99-418c-93cc-fcd189eac053 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.818966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcb969-9f8f-4a24-b6ee-5530d521543a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.862891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.863549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.863825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.864243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.865283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] RBAC: Adding query filter params (interface=public, service_id=d8660961b37043ec887a5a5b7a9f1e4c, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:48.865594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.867242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8040355-6c61-4381-8199-fe50b57c593b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.877656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 385/770] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/endpoints?service_id=d8660961b37043ec887a5a5b7a9f1e4c&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:48.878987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.879270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.879988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.902917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:48.904581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:48.907959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83eba3-9d99-418c-93cc-fcd189eac053 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:48.930973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.945498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:48.959449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:48.960357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:48.960863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:48.961347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:48.970334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:48.972281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:48.986287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:48.987245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:48.988120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:48.993231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 386/771] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:48.993231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:48.997574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 386/772] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:49.003521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.104857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df712-25cb-4b5c-a462-3433b2713cdc None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.144762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.145298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.145573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.145844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.146464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.147842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3537-65b8-4a1f-9d22-2f07452a3f06 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.154233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 387/773] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:36:48 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:49.155640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f0f-297c-4a8a-b3da-befaa4fc4342 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.155925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f0f-297c-4a8a-b3da-befaa4fc4342 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.156197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f0f-297c-4a8a-b3da-befaa4fc4342 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.157861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 388/774] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:49.170358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.170672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.170903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.196704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:49.197941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:49.210027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:49.282531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cc622-252c-4538-b7b1-20637f4ab8d9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.300089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 389/775] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:36:49.324286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.325307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.325788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.326258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.327144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:49.327719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.329416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.343018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 387/776] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:49.348770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.424378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.477464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdef40-f325-49b0-b7c8-14646c4ea6e4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.483838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:49.531938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.532677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.534947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.534947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.536543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=d882cbdd492344c7a3638c65ed08b32d, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.538005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.561541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f22db-7650-449f-bbe8-aa1036a79a43 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.598770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.599464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.599746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.600016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] PATH_INFO: `/v3/services/network` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.600598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.602097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:49.612195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:49.613757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:49.614614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 388/777] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:49.616074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.635513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 390/778] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:49 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:49.641268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.657185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:49.733599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423186-be6f-45d3-81e2-c0aae17e6a78 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.743356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:49.773313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.774446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.774960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.775442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.788800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:49.792895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d32af-edb7-4ce4-a22f-9b490a0dac12 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:49.794352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:49.795398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:49.796269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:49.803704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 389/779] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:49.806162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.832207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.833055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.833169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.833381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.834129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:49.834430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.835871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.841167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 391/780] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:49.847795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.924361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb5953-9c9a-41e3-88f6-834e969e9fcb None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.924561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:49.960764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8787a30-077d-47fc-ac5c-82e32b3646c7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:49.972606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:49.973523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:49.973985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:49.974450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:49.975307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:49.975790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:49.977428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:49.988164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 390/781] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:49.995536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:49.999787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.000405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.000712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.000989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.002464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] RBAC: Adding query filter params (type=network) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:50.002464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.004164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.010820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 392/782] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:50.016718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.030756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:50.111226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24cf4-e397-46b1-b766-f7cbdbe05786 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.122111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134660e-a9fd-4bb2-a9da-edd95b3beedc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.148895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.149893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.150401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.150877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.158297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.158910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.159188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.159455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.160219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] RBAC: Adding query filter params (interface=public, service_id=8dba7093e0c0479e93730ea1d667e4c1, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:50.160534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.161935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3f665-ae1b-4d55-bc99-92d9697918fb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.164488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.166195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.167551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 393/783] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/endpoints?service_id=8dba7093e0c0479e93730ea1d667e4c1&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:50.172959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b385-1af3-476c-a85e-594ea7d6a3fb None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.173256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b385-1af3-476c-a85e-594ea7d6a3fb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.173564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928b385-1af3-476c-a85e-594ea7d6a3fb None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.175223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 394/784] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:50.175471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:50.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b385-1af3-476c-a85e-594ea7d6a3fb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-764d491b-0d0a-4269-84f7-e817eb2b92b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:50.184318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:50.194579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 391/785] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.195673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.214782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:50.216550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:50.292240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928b385-1af3-476c-a85e-594ea7d6a3fb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.324193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 392/786] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:50.325786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.328490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.329252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.329592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.330548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.331283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.333513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.338561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 395/787] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:50.345062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:50.483162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e263d4-9f65-4362-bdcc-42d9247a4ec7 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.532945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.533575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.533856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.534636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.535304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:50.535608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.537066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.553476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 393/788] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:50.556970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.628266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:50.652109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.682726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb6a0-0d77-4d0f-8946-78a33e0bed86 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.742968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.743526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.743804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.744072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.754950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:50.759963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:50.763363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:50.780583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:50.781568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:50.782518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 396/789] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:50.790936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:50.843880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836d9f6-3b52-4d29-ac3d-174b259e853d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:50.922143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:50.923216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:50.923639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:50.923974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:50.951354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:50.957306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:50.958297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:50.959112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 394/790] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:50.965671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.110767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffa93e-3263-4c55-854c-d30bcde0d398 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.138662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3bb01-376c-4d6f-8614-cf5216ff33df None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.156499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.157155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.157441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.157727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.158944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:51.159255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.159784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.165359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.180775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 395/791] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:51.183163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.222600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.285968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.286599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.286947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.287226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.287968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:51.288274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.289783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.314618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f4b25-cd66-4847-ab40-71d1d22ebcbf None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.318182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 397/792] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:51.320285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.430788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.431430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.431708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.431982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.439352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7481b7b-7ead-4a3c-9ff7-995978cba401 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.465732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.467217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.478718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.479465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.479747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.480017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.491959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.493660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:51.498230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:51.499229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:51.500107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:51.500107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:51.500107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:51.504810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 396/793] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:51.510001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:51.510925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:51.511783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:51.511783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:51.511783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:51.512846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.515854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 398/794] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:51.517678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.559594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.611363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.664764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1dd835-7d4f-44b6-9bee-f6156cc72f04 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.671203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ecf87-4a51-4d30-805e-c9d6482bd40a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.709876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.710231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.710488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.710749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.710862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.711028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.711122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.711295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.711800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:51.711966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:51.712120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.712208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.713579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.713692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049971b-4a86-47e7-a98e-95b7d7dfc0c5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.727780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 397/796] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:51.729207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b462d-e5d3-4471-beca-507f6bdfa78e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.729487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b462d-e5d3-4471-beca-507f6bdfa78e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.729753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b462d-e5d3-4471-beca-507f6bdfa78e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.731902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 399/797] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:51.732901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.735135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 398/797] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:51.736545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.736852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.737125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.756687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:51.757990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:51.843037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 399/798] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:51.844029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:51.853516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.862942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0491-b595-49a4-ae54-24dbc1393a12 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:51.906245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:51.907149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:51.907429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:51.907702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:51.908879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:51.919736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:51.919736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:51.931920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:51.934072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:51.935508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 400/799] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:51.938702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.056172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d172ba-fba1-4865-9371-813a047fa806 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.100174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060801c-a7c2-41f2-8ef2-746d5f06b66d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.103332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.103955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.104232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.104502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.105265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:52.105569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.107001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f1e2b-e591-46ba-a405-4135383b154b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.129961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 401/800] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:52.131561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68a61-27ef-4ce0-9577-4348a2141c35 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.132016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68a61-27ef-4ce0-9577-4348a2141c35 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.132470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d68a61-27ef-4ce0-9577-4348a2141c35 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.138025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 402/801] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:52.138857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68a61-27ef-4ce0-9577-4348a2141c35 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.151057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.151546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.151830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.152103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.156007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.160780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.162290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-4aac6f7a-916d-4e5e-b86b-f0a8d8861f07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:52.173913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:52.174814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:52.175725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 400/802] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:52.176988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.176988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.177237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.198075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:52.199279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:52.270862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d68a61-27ef-4ce0-9577-4348a2141c35 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.272191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 401/803] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:52.272896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.298881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.317846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.318780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.319061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.319330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.333103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.335066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:52.355610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:52.356668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:52.357558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:52.357558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:52.357558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:52.359613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 403/804] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:52.360729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.406244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d107bca-bee7-4230-ad72-621f3ea4507b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.445404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.446327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.446750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.447188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.448025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:52.448470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.450079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.455690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.465038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 402/805] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:52.466146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.503386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760a93-92be-4933-93ab-e02c3cf02233 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.541237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.542256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.542789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.543274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.544102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.545741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:52.555729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:52.556825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 07:36:52.557647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:52.558360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 404/806] 10.222.0.72 () {56 vars in 1129 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:52.558965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.574096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.592609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554aff-09b7-4c94-ae94-85b24b76f92e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.629490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.630411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.630893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.631344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.632059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:52.632356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.633731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.645833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 403/807] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:52.646783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.669803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1506d-c7da-4f96-8461-616e5a1b52e6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.697652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.708636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.709332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.709605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.709875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/e977eb2e34a945b4adeffbc92d154248/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.712077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=e977eb2e34a945b4adeffbc92d154248, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.713577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.755460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33471fe6-6f81-4bc6-a417-cf27b9db78f3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.788945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.789578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.789851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.790119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.800533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:52.806069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:52.807143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:52.808050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 404/808] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:52.808838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.811405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:52.822735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 405/809] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:52 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/e977eb2e34a945b4adeffbc92d154248/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:52.823713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.921533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14acc7-295f-482e-ad41-5bd2f9cee7ea None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.950659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a1990c-210d-44e2-a375-0d2ac13b8000 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:52.968399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.969401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.970308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.970308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.970561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:52.970866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:52.972238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd83fc-099b-4fb4-888a-b5856a117b76 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:52.977477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 405/810] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:52.978657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df9c1-7998-42ba-8e40-3ecc5a42c783 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.978946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df9c1-7998-42ba-8e40-3ecc5a42c783 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.979221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51df9c1-7998-42ba-8e40-3ecc5a42c783 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:52.980571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 406/811] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:52.981418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df9c1-7998-42ba-8e40-3ecc5a42c783 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:52.992564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:52.993231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:52.993511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:52.993779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.001055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.002578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:53.013812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:53.014710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 406/812] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:53.016254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.067634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.074523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.106949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51df9c1-7998-42ba-8e40-3ecc5a42c783 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.139649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488215b-2cbf-4ba4-aeeb-c9cff73b7a05 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.145163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.145774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.146056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.146325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.146989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:53.147604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.148645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348d31-4baf-4441-b9fe-4420422d00fa None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.157263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 407/813] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:53.158620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.158906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.159178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.177880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:53.179068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:53.182478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.182478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.182704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.183063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.183767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:53.184077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.185503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.196128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 407/814] 10.222.0.72 () {56 vars in 1124 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:53.196128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.264925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 408/815] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:53.264925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.310591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74dc29-5df7-43d4-85d9-67ab4eeb52cd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.325612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.334469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.354775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.355657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.356085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.356504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.357371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:53.357841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.359489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.374807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 408/816] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:53.376005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.383686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd34d4c-f149-4cea-bc71-29a2f48709ce None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.429511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.430147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.430419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.430685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.441759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.443376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:53.460020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:53.461023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:53.461906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:53.461906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:53.461906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:53.461906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 409/817] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:53.462561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.490275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2634d1c5-517b-4691-82b2-5273cfe3f4ab None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.528203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.529172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.529783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.530295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.531202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.533331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.569470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9473036-fe0d-4eb5-af18-560497d6e3a8 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.582529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.603645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.604335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.604606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.604898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/0c877a807efb45a8b6475e0c8db99e98/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.618770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=0c877a807efb45a8b6475e0c8db99e98, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.620256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.657349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 409/818] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:36:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:53.658925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.693060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 410/819] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:36:53 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/0c877a807efb45a8b6475e0c8db99e98/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:36:53.694642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.698642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:53.817335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe165-9073-430d-a0dd-994c7d1764d3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.872798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.873778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.874054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.874329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.875311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:53.875616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.878476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.887882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f4b98-dd2c-4668-b473-498187b1fe6b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:53.896566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 410/820] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:53.904753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:53.948610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:53.949453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:53.949734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:53.950004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:53.950594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:53.952220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:53.969420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:53.970614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 07:36:53.971519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:53.974262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 411/821] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/services/network => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:53.979981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.015597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:54.030730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:54.049391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad608f8-f6fe-4cf5-a264-b5e49cf7021c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.119820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.120717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.121138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.121572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.122202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5c36a-9382-47df-bc2e-46e0de7459a9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.137778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.139488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:54.161136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:54.171710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:36:54.172910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:54.173910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.173910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.174079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 411/822] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:54.174160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.174160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.174160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] RBAC: Adding query filter params (name=network) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:54.174160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.174160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.174544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.174636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 412/823] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:54.180060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.369019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:54.369429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75c85d-e44b-4d5c-866d-0b2632b65f2a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.377243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00234ae-c560-44e5-b145-5431e8127b71 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.450208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.450815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.451088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.451352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.452083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:54.452382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.453837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.465279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.465901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.466173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.466437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.467065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] RBAC: Adding query filter params (type=network) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:54.467360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.468736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.484594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 413/824] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:36:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:54.485509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.489708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:54.503425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 412/825] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:36:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:54.515065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.607277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc90921-efae-4951-bb6d-beefea5c5f33 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.647361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.647885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.648156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.648419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.649103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.650942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.693533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4929a2f-88e4-4362-afef-0c55ce8a1bc6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:54.732130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:54.732971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:54.733164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:54.733412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:54.771714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 414/826] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:36:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:36:54.807893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:54.810442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:54.811942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876d46c-b018-4594-8e43-83461cddcc8e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:54.920721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 413/827] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:36:54.927082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:54.954672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.059866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342975b-2bba-4abd-881a-d93eaa46416d None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.060155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342975b-2bba-4abd-881a-d93eaa46416d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.060456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342975b-2bba-4abd-881a-d93eaa46416d None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.062221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 414/828] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:55.069166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.069881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.070607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.089112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362622a-3c2d-403e-abdc-1fbdf62e637e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.092099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:55.093543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:55.132571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.133235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.133520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.133792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.149286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.151000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:55.169459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:55.170412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:55.171342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:55.171342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:55.171342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:55.176630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 415/829] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:55.179623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.181366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 415/830] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:55.243788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.289385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547031b-a3c7-4aa0-9356-d2f481719f28 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.325577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.326535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.327008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.327470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.328376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:55.328921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.331138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.349094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 416/831] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:55.354732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.396339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.467658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fd144-419c-4578-88d6-da76ce9e9aba None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.502201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.507639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.508533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.509062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.509527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.517190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.518964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:55.530946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:55.532114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:55.533755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:55.535848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 416/832] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:55.541716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.640444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96309fb0-6324-4b5d-973f-2911e90cc0c9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.641304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.656588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8134aa09-3eb7-47e5-93f4-7b4b0ae821fc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.663743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.683697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.684401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.684696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.684968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.697618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:55.702445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:55.703370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:36:55.704198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:55.708597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 417/833] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:55.708969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.709427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.710004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.710591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.711070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.711971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:55.712842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.715530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.738283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 417/834] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:55.740356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.839874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12468af5-9425-44a8-b6c4-2cc6c24d9699 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.850601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41652d11-e2f3-4e5a-b50f-e6859a1d11f1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:55.876841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.877841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.878315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.878778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.879635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:55.880146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.881736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.887087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:55.887752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.888028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.888672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.889049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:55.889355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:55.890855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b61ad6-ea69-4376-9cd8-daf9ab1acaad None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:55.893420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 418/835] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:55.899371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:55.908116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 418/836] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:55.949710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6f4d-ee86-4c84-a9d4-42030ad2e002 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.950020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6f4d-ee86-4c84-a9d4-42030ad2e002 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.950263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6f4d-ee86-4c84-a9d4-42030ad2e002 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.951898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 419/837] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:36:55.956047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:55.958174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f567392-fdb1-443f-9408-13c32b0ef55e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:55.958458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f567392-fdb1-443f-9408-13c32b0ef55e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:55.958739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f567392-fdb1-443f-9408-13c32b0ef55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:55.982347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8f567392-fdb1-443f-9408-13c32b0ef55e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:55.983585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8f567392-fdb1-443f-9408-13c32b0ef55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:55.997397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.033190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf4ccd-6bc3-497f-9e29-74fa1c05945e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.068239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 420/838] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:36:56.069710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa51f-f231-4407-964c-b5958e4eb06a None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.070526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa51f-f231-4407-964c-b5958e4eb06a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.070526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5aa51f-f231-4407-964c-b5958e4eb06a None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.071946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 421/839] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:56.080538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.080773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.080938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.081665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.081774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.081863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.081863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.101198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.103482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:56.103605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.105167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:56.124307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:56.125423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:56.126929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:56.126929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:56.126929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:56.128811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 419/840] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:56.134567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.196593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 422/841] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:56.261224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.273869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0101b-4003-4ae0-9dcf-b16215ceff93 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.310693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.311303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.311582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.311856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.312510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:56.313360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.315327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.327926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 420/842] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:56.337113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.434292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.463792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.507893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2224e79-ca54-471a-af86-ff502322a32b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.554601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.555248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.555609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.555851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.565334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.566788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.572313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cc90c-3c5f-4fbe-9f51-47fe37f41483 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:56.578850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:56.579838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 423/843] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:56.580895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.623260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.623900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.624175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.624443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.625211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.628043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:56.641317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:56.642381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:56.643725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 421/844] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:36:56.644783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.664702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.704373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39407d35-26d6-45ca-85c2-01761e2a4b50 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.741656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.742323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.742562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.742746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.744704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.753518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6618-b0d0-4d2e-bd81-0e3cde2ca52b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.755279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:36:56.759848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:56.760793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 424/845] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:56.762352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.799018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.800074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.800583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.801341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.802681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:56.803205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.805596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.828687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 422/846] 10.222.0.72 () {56 vars in 1135 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:56.828687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.873806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae280a-4bfe-491b-89e2-833238de9696 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.899139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:56.910997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.911601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.911877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.912145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.912798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:56.913097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.914609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.924133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 425/847] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:56.924133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.929176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54632b-691f-4e04-85f3-c73e79345840 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:56.965765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:56.966393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:56.966628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:56.966898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:56.967561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:56.967864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:56.969271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:56.979328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 423/848] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:56.979929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:56.987852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.031565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94ba08-26d3-40b2-b2af-78694a5d81aa None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.071781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.072304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.072583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.072882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.087192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51d864-bedd-416a-a106-d0205ae24875 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.124630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.125618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.126151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.126395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.126992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:57.127295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.128714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.131069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.144684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 424/849] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:57.144684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.160452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.162052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.228307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.276688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76a23-88b3-429b-8621-7a82de87bd72 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.276887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 426/850] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:36:57.281028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.325287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.325907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.326188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.326462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.339639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.341171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:57.357853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:57.358834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:57.359711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:57.359711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:57.359711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:57.359711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 425/851] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:57.365808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.411319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe832f38-e2b8-49a7-a9c8-991eda71c746 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.432381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.447895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.448544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.448864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.449121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.460170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.461738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:57.478947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:57.479920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:57.480844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:57.480844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:57.480844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:57.480844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 427/852] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:57.484054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48361-612c-4d9f-bbf5-241991d08add None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.485833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.498603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.526432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.527050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.527323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.527587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.528234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:57.528532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.530008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.544076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 426/853] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:57.556690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.608862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dfc1b-c3d0-4635-9bab-23327d94e427 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.648733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.649442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.649786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.650146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.650858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:57.651161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.652523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.665148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 428/854] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:57.665684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b27c9-0fb3-4656-bc72-0df5a355dae6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.673695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.680481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.705636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.706270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.706546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.716377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.717864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:36:57.735240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:57.736292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:57.737862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 427/855] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:57.742074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.762767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.796169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cff70-2d2b-4cb8-8743-956c4bc39c38 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.833751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.834390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.834692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.834989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.841926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.843462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.852076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c645f-66b4-42b0-805a-0c9f9f6bdb80 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:57.855407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:57.856277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:57.857195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:57.860963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 429/856] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:57.862436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.893719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:57.894763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:57.895241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:57.895708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:57.896677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:57.897183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:57.898864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:57.918165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 428/857] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:57.925412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:57.964694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:57.990416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a2869-ad75-4767-ae23-bf9cc8c33cb4 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.028406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.036894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.037637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.037968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.038296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.039612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:58.040061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.043168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd428ea-970c-4f7b-9e69-88143d7d0630 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.055079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9166c75-c461-4858-98ce-38129cfc3be6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.056544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 430/858] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:58.061999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1cf1-ab37-449c-8322-67f01e05a4d4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.062277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1cf1-ab37-449c-8322-67f01e05a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.062542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1cf1-ab37-449c-8322-67f01e05a4d4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.064081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 431/859] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:58.065481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc1cf1-ab37-449c-8322-67f01e05a4d4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.105875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.106494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.106778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.107048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.177258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc1cf1-ab37-449c-8322-67f01e05a4d4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.182540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.184104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.213282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.213926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.214201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.214415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.214528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.215191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=e977eb2e34a945b4adeffbc92d154248) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:58.215499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.217044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13641c-c6ef-44e3-95ae-5fdc0dd09da0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.232633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 432/860] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/role_assignments?user.id=e977eb2e34a945b4adeffbc92d154248&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:58.232633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.232633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.232633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.245858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:58.247239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:58.318978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 429/861] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:36:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:36:58.320102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.323773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.352839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 433/862] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:36:58.449053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fe9b-dad3-49ae-a4f1-5f9e4514f533 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.479510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.486009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.486914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.491603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.491603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.498277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.499771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:36:58.515317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:58.516298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:36:58.517835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:36:58.517835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:36:58.517835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:58.517835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 430/863] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:58.522619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.626553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fa571-0e11-4782-8347-c1884362d0b7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.632232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.662371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.663037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.663366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.663590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.664241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:58.664546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.665973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.678451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 434/864] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:58.683898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.737665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.747049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ec2db-ef7b-4e4f-81e0-f3b953493133 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.784013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.784764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.785046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.785320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.785915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.787463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:58.797520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:58.799219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:36:58.800089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 431/865] 10.222.0.72 () {56 vars in 1132 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:58.805378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.809846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cb331-9d79-4a37-9361-728aa09a3490 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.855992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.857592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.858069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.858531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.865484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:58.870518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.872678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:36:58.886720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:36:58.887649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:36:58.888524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:36:58.892691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 435/866] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:36:58.894015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:58.938070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499c44a-ec87-41cf-afff-8eef15ec2e77 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:58.983031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:58.984289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:58.984817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:58.985954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:58.987078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:58.987589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:58.989244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:58.996265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 432/867] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:59.002090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:59.003110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818759f-73a2-43f0-8b5c-d893c5e4e16f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:59.006113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:59.052906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:59.054498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.054975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.055438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.056803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:59.057789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:59.059729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:59.077645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 436/868] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:36:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:36:59.084091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:59.131564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef912b-b366-42de-81d0-002436578a12 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:59.145722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:59.175350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:59.176217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.176496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.176800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.177432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:59.177735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:59.179143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f2c26-ce0e-43e0-b1cd-14909b1e8f85 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:59.183641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 433/869] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:36:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:59.185032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd63aa-d5b8-4554-8fc4-81b5c7a575d5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.185143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd63aa-d5b8-4554-8fc4-81b5c7a575d5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.185379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd63aa-d5b8-4554-8fc4-81b5c7a575d5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.186609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 434/870] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:36:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:36:59.198889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e38f1-0e37-427a-85a3-6724cfefe5d9 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:59.199157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde729e-128e-400f-9788-f29db977d8e2 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.199588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde729e-128e-400f-9788-f29db977d8e2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.200012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde729e-128e-400f-9788-f29db977d8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.220522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4fde729e-128e-400f-9788-f29db977d8e2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:36:59.222029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4fde729e-128e-400f-9788-f29db977d8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:36:59.240485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:59.241212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.241737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.242108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.243072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=0c877a807efb45a8b6475e0c8db99e98) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:36:59.243484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:59.245348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:59.252930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 437/871] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:36:59 2022] GET /identity/v3/role_assignments?user.id=0c877a807efb45a8b6475e0c8db99e98&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:36:59.309089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 435/872] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:36:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:36:59.583816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:36:59.613129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:36:59.689301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159cec4-1538-4f9b-be97-d2e44fa56bdf None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:36:59.725176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:36:59.725802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:36:59.726080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:36:59.726351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:36:59.727012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:36:59.728381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d802c80-504c-4529-8b80-9e5054a6a349 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:36:59.807117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 438/873] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:36:59 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:00.302872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be458c7-7a27-41cf-956d-bb6f937faaad None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:00.303400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be458c7-7a27-41cf-956d-bb6f937faaad None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:00.303741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be458c7-7a27-41cf-956d-bb6f937faaad None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:00.305643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 436/874] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:00.316199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:00.316774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:00.317230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:00.338865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:00.340062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:00.421072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 439/875] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:00.686708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be458c7-7a27-41cf-956d-bb6f937faaad None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:00.798182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be458c7-7a27-41cf-956d-bb6f937faaad None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:00.801812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:00.834121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:00.835047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:00.835526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:00.835964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:00.836758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:00.838302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:00.849004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:00.850053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:00.850887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:00.853270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 437/876] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:00.861241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:00.885035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:00.965429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1dc36a-d39d-49de-805c-f6dfad9ab44d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:01.000892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:01.001497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.001773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.002043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.002670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:01.002974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:01.004391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1360-23ee-498a-88a2-c2c04ab577cd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:01.009367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 440/877] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:01.015075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:01.121725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaacef-2f6c-4643-b27f-22f948afb2e1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:01.161270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:01.161927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.162203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.162512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.163150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:01.163445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:01.164893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd0d5-0f7c-4415-bbd4-56d53a6d305d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:01.169959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 438/878] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:01.180563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be3c1-d1bb-4aa8-b813-3cd66fdc58a2 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.180790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be3c1-d1bb-4aa8-b813-3cd66fdc58a2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.181032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676be3c1-d1bb-4aa8-b813-3cd66fdc58a2 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.188803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 441/879] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:01.193212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.193212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.193212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.219230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:01.221638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:01.261435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:01.270838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a83-678d-4ee7-92f7-16da9a0ae843 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.275253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a83-678d-4ee7-92f7-16da9a0ae843 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.275253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b30a83-678d-4ee7-92f7-16da9a0ae843 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.275253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 442/880] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:01.278550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da870-118f-4344-be19-24dbc713fc68 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.278830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da870-118f-4344-be19-24dbc713fc68 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.279099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157da870-118f-4344-be19-24dbc713fc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.301944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-157da870-118f-4344-be19-24dbc713fc68 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:01.303174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-157da870-118f-4344-be19-24dbc713fc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:01.322606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 439/881] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:01.388760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 443/882] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:01.608739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:01.667195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157da870-118f-4344-be19-24dbc713fc68 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:01.678070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:01.718455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19595eb-ba28-4f9d-bee3-29a15cf53f73 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:01.757667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:01.758618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.759136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.759571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.760382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:01.761942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b223a-b878-4884-b5b2-074d249e25d9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:01.791742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157da870-118f-4344-be19-24dbc713fc68 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:01.828026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:01.828675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.828982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.829259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.839144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ce6be906-3d4f-42d6-8b70-de24d4c467f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:01.843810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:01.845312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:01.846196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 444/883] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:01.846877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2054-a22b-4967-87e7-050cb5661693 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.846877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2054-a22b-4967-87e7-050cb5661693 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.847041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f2054-a22b-4967-87e7-050cb5661693 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.848689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 445/884] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:01.850600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f2054-a22b-4967-87e7-050cb5661693 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:01.871145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 440/885] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:01 2022] POST /identity/v3/services => generated 269 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:01.871145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:01.871779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:01.872249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:01.894782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:01.896450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:01.916439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:01.939045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:01.988281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f2054-a22b-4967-87e7-050cb5661693 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:01.996784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 441/886] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:02.036441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.037108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.037391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.037669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.039408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.039728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.041197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.051054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 446/887] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:02.056597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.164731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83bcbe-7261-41e9-87db-2b474ea79a70 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.199900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.200527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.200825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.201091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.211521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.213001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.222260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:02.228587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:02.229640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:02.230903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:02.230903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:02.230903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:02.230903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 442/888] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:02.236522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.266877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.364965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04d939-e54f-4649-b5fa-ba03f9438c5e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.394441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.404432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5b145-119a-48cf-bc9f-9f3429892ffe None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.407709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.407709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.407709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.407709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.407999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.407999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.410021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.431453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 447/889] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:02.433648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.448717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.449536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.449815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.450084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.450736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.452259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:02.461888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:02.462878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:02.463731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:02.467989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 443/890] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:02.469089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.539580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.554685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb13719-10d5-41d1-995a-404abd84c40a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.582674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f45ade-210f-40ec-92e6-e8655900246c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.591670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.592617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.593116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.593598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.602069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.603820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:02.616544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:02.617541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:02.618462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:02.625254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 448/891] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:02.626454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.630567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.631693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.631807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.631994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.632633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.632960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.634323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.645148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 444/892] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:02.646309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.678837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.737094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48db81d-e775-4590-acaa-180e2eaa1473 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.760838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151d3b7-2adc-4ab1-a6c9-8cf5b0b874cc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.773352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.774080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.774449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.774814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.775635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.776049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.777596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.791320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.800812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 449/893] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:02.802418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.804737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.805718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.806137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.806691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.807534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.808068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.809660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.821034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 445/894] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:02.822136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:02.911578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79c79-8ae6-4d45-a2fc-272e765d9778 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.914066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:02.938715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5be2-09c9-4297-aee1-8fb995ecc51d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:02.947870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.948475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.948794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.949062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.979066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:02.980013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:02.980505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:02.981094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:02.982106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] RBAC: Adding query filter params (interface=public, service_id=d1a9b9d54b224908bdcae6a656247a8e, region_id=RegionOne) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:02.982696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:02.984263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:02.998028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 446/895] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/endpoints?service_id=d1a9b9d54b224908bdcae6a656247a8e&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:02.999306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.020951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.022410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.063998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.118582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe5ecad-9909-417d-a476-8ccf4b2cb9eb None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.132981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 450/896] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:37:03.134219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.137202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.158452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.159376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.159827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.160297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.161148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.162714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512470a8-2b81-4793-a2ff-6821f103778f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.177049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 447/897] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:02 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:37:03.187109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc106d7b-557b-4921-9094-aed9935fa59a None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.187727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc106d7b-557b-4921-9094-aed9935fa59a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.188379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc106d7b-557b-4921-9094-aed9935fa59a None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.190228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 448/898] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:03.197689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.198035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.198270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.220763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:03.222465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:03.259638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b74d-98b7-4288-8415-d706cef38319 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.305093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 449/899] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:03.307933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.308691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.309059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.309396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.321214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.322796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:03.339232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:03.340206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:03.341130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:03.341130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:03.341130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:03.341130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 451/900] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:03.346909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.400061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.452949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6f50c-5865-4c6f-913b-614d7e7d84de None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.490702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.491360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.491739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.492012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.492696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:03.492972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.494338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.506372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 450/901] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:03.511908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.570605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.573536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.630754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5775998-c9fb-4bdd-8599-3bceff243804 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.668448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.669121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.669402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.669674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.677223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.678672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:03.691695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:03.692613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:03.693868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:03.697757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 452/902] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:03.698641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.707157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24df16e-1d6e-4050-a431-3765d2b101e4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.721698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.747400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.748065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.748349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.748621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.749374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.751415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.816038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602c46f-2496-453b-aa18-f58e9600a29a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:03.833324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 451/903] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:03 2022] POST /identity/v3/services => generated 241 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:37:03.842642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:03.871996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:03.873051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.873523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.874008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.874972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:03.876068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:03.877871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a90a4-9ab8-4bbf-9d28-a9fbf9e50810 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:03.893508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 453/904] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:03.901387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:03.976675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf955c-ea7f-412e-97db-908d645987dc None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.977022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf955c-ea7f-412e-97db-908d645987dc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.977227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf955c-ea7f-412e-97db-908d645987dc None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:03.978869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 454/905] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:03.984476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:03.984808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:03.985088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:04.004779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:04.006294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:04.024001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81335698-bf65-484e-bfd2-a0cac713d3ef None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:04.069263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:04.069774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:04.070047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:04.070316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:04.071136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] RBAC: Adding query filter params (role.id=039cf3c7d1ae4cedab5ed8f9b894aaad, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=0c877a807efb45a8b6475e0c8db99e98) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:04.071344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:04.072810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:04.078836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 452/906] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:03 2022] GET /identity/v3/role_assignments?user.id=0c877a807efb45a8b6475e0c8db99e98&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=039cf3c7d1ae4cedab5ed8f9b894aaad => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:04.092969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 455/907] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:04.383383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:04.421360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:04.493230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220a52-2f1b-47c3-b6b9-8a455859a787 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:04.529235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:04.529871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:04.530148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:04.530419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:04.531010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:04.532408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:04.541505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:04.542449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:04.543369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 453/908] 10.222.0.72 () {56 vars in 1132 bytes} [Wed Aug 24 07:37:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:04.550816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:04.659768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256946d-e127-4f09-b80a-9f5d0dbb32be None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:04.661572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:04.696574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:04.697543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:04.698036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:04.698497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:04.699336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:04.699831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:04.701416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:04.707426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 456/909] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:04.713614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:04.814958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:04.821343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf909de-3666-43f9-8212-bbcb3d679375 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:04.865889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:04.866521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:04.866804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:04.867078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:04.867725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:04.868027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:04.869431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:04.887194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 454/910] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:04.898451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:05.063985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3eac8-d693-4a02-8cf2-c998b2e05aee None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:05.120008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:05.120695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.120983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.121289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.122004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] RBAC: Adding query filter params (interface=public, service_id=6d6ac72942414a85bc14ee36efd72641, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:05.122310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:05.123680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f48eb7-f9c1-4e61-9695-2b45a2d80c65 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:05.130335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 457/911] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:37:04 2022] GET /identity/v3/endpoints?service_id=6d6ac72942414a85bc14ee36efd72641&interface=public®ion_id=RegionOne => generated 194 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:05.135406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:05.232932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ba5-a9fc-4a1f-b880-abbab926d4bc None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.233257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ba5-a9fc-4a1f-b880-abbab926d4bc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.233480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ba5-a9fc-4a1f-b880-abbab926d4bc None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.235685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 458/912] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:05.240624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.241114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.241390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.273232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:05.274430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:05.294943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699863fc-ccb0-4efb-a3dc-e9ca57741605 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:05.345846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:05.372084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:05.373824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.374104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.374373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.376587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:05.378836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:05.397391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 455/913] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:05 2022] GET /identity/v3/services => generated 1497 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:37:05.426120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 459/914] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:05.702715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:05.731610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:05.808064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1935-2553-47bc-aa23-4bf5054e3ddd None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:05.844184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:05.844851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.845125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.845391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.845973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:05.847334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-114edc53-24ff-43e6-959b-dc4273014992 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:05.858356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:05.859379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 07:37:05.860804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:05.861526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 456/915] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:05.871589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:05.918104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985390-bef9-422d-a2da-0d0ca4565e51 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.918401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985390-bef9-422d-a2da-0d0ca4565e51 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.918656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985390-bef9-422d-a2da-0d0ca4565e51 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.920238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 457/916] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:05.926215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56482f0-2e46-4519-9af1-19ea103c3811 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:05.926494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56482f0-2e46-4519-9af1-19ea103c3811 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:05.926765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56482f0-2e46-4519-9af1-19ea103c3811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:05.948736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c56482f0-2e46-4519-9af1-19ea103c3811 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:05.949957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c56482f0-2e46-4519-9af1-19ea103c3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:05.982292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff707838-b859-4542-a93d-1f24ea0260cf None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.017347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.018104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.018453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.018793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.019503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:06.019879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.021340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.026656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 460/917] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:06.033598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.033735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 458/918] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:06.062357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.143097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7219c9-af19-49f8-8ccf-686aafb7ee36 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.170341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef904-266b-4217-93cf-43607dc535cd None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.170618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef904-266b-4217-93cf-43607dc535cd None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.170890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ef904-266b-4217-93cf-43607dc535cd None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.176310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 459/919] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:06.178677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.178955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.179229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.179327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.179886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.180164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.180433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.181087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:06.181389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.182861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.188065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 461/920] 10.222.0.72 () {56 vars in 1114 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:06.194512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.197966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:06.199276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:06.218557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.285010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 460/921] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:06.302075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.308769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d874c28-4cd9-40e7-ac22-2e7e6a3467f2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.331225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.345025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.345657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.345930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.346242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.346901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.348253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.444028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 462/922] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:37:06.475878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520d671-25c0-402b-83b8-cb616b10e725 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.525968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.526632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.526979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.527253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.527866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.529312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:06.539940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:06.541015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:06.542017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 461/923] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:06.550101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.574914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.593345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.683196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc673ea4-7692-467a-9234-71e971bb1d55 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.702274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba929d-4193-438d-8743-b76c4cb0657e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.720594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.721431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.721848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.722258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.723045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:06.723487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.725032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.730453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 463/924] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:06.736251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.744771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.745411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.745669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.745933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.753090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.761108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:06.768311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:06.769271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:06.770174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 462/925] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:06.775734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.872900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:06.874399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbcf0bd-b1b2-4951-a505-3dfae0e0ecfa None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.910122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee29ee4-4aee-487f-a001-df08a9782429 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:06.920637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.922070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.922580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.923055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.924186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:06.924880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.926936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.934356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 464/926] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:06.948684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:06.955081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:06.955733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:06.956021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:06.956293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:06.957046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:06.957352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:06.958784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:06.967852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 463/927] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:06.973935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.026075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.058631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adc970-96ba-4ce3-af33-03ef2ad6b3a1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.096591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.097632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.098096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.098580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.099456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] RBAC: Adding query filter params (interface=public, service_id=2039940db07d47c9abe0297996276bd8, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:07.100047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.101884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.113280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559be681-dc0f-41b9-8e03-d1f8932dec38 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.113700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 465/928] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/endpoints?service_id=2039940db07d47c9abe0297996276bd8&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:07.115105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.146375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.154162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.155155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.162474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.162474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.169981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.172539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:07.192609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:07.193662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:07.194996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:07.194996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:07.194996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:07.194996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 464/929] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:07.199339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.239541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80707e92-3785-4506-9d80-3d8975a8a56a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.276599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.277782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.278360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.278832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.279636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.281509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.283920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.296097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 466/930] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:37:07.329562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6f96b-c4a2-436f-b1d7-3f85b2ed7d1d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.369577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.370200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.370476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.370748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.371402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:07.371704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.376883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d46dc-3440-4900-9b94-facbea63449f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.394197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 465/931] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:07.408307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.437118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ab017-e9a5-4e0b-a4dc-0294c1d54c4e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.437388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ab017-e9a5-4e0b-a4dc-0294c1d54c4e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.437706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ab017-e9a5-4e0b-a4dc-0294c1d54c4e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.439571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 466/932] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:07.444920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.445201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.445477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.465489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:07.466685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:07.529024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.538309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff606-f377-4356-959f-ab92b91e9a54 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.544366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 467/933] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:07.575413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.576191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.576572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.576966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.585087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.586643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:07.598792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:07.599749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:07.600837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 467/934] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:07.607803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.717747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94745d5c-8e5c-47ab-bdb0-e9b7d2bc214c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.753635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.754247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.754556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.754836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.755565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:07.755871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.757538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.776316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 468/935] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:07.783547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.818714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.852383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.852943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:07.912024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6907e-8bb8-4604-9438-8eab1dcc1a04 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.928278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc29c-54f1-4881-aa60-2a83917892e1 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:07.949836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.950473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.950755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.951026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.962487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.963958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.967456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:07.968445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:07.969548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:07.970120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:07.971202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:07.973006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:07.983181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:07.984382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 07:37:07.985439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:07.987126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:07.988048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:07.988947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:07.988947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:07.988947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:07.989968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 468/936] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:07.990658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 469/937] 10.222.0.72 () {56 vars in 1132 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:07.997650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:07.998401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.099498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:08.103828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff40a5-d322-4cbc-bf16-5cd5c36a031b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.106696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f2100-101e-4fd4-acf3-195e84425b01 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.125750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:08.139796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.140405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.140750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.141537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.142187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:08.142494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.143860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.147601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.148530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.149055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.149551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.150501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:08.151020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.152581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.156236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 469/938] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:08.163221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 470/939] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:08.163806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.164090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.268949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159964c-37af-416e-a569-3c381cdf4286 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.289952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e0d58-b087-4d38-881b-caeea814d4d3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.304863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.305386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.305647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.305916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.306550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:08.306859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.308229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.318435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 471/940] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:08.319319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.334361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.335508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.335981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.336511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.344503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.345409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:08.347222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:08.359414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:08.360908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:08.361778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:08.367849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 470/941] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:08.370342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.420639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:08.442382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6f950-bd44-4a85-ba33-397240e1198a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.492084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.493369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.493979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.494540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.495385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.497617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0e066-9de3-46cf-a5f1-ebf803bb1ab4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.500713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1271ee9-c454-49a2-bf76-d5923659bbc7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.550793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.550793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.550793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.550793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.551262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:08.551262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.551262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be4ff9-03ba-4fd9-bfc6-349d1a3084bc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.554924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 471/942] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:08.560117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f25dcf-0be0-42c6-8326-9d9d50c105e4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.560117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f25dcf-0be0-42c6-8326-9d9d50c105e4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.560117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f25dcf-0be0-42c6-8326-9d9d50c105e4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.560117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 472/943] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:08.561073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f25dcf-0be0-42c6-8326-9d9d50c105e4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:08.635118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 472/944] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:08.636812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.637377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.638332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.664182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:08.666720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:08.666870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:08.678864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f25dcf-0be0-42c6-8326-9d9d50c105e4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:08.726204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:08.727549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:08.727829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:08.728105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/0c877a807efb45a8b6475e0c8db99e98/roles/039cf3c7d1ae4cedab5ed8f9b894aaad` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:08.731459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=0c877a807efb45a8b6475e0c8db99e98, role_id=039cf3c7d1ae4cedab5ed8f9b894aaad)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:08.733678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300696f2-30ec-486f-98d3-6de660ec88b0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:08.761666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 473/945] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:08.807303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 473/946] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:37:08 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/0c877a807efb45a8b6475e0c8db99e98/roles/039cf3c7d1ae4cedab5ed8f9b894aaad => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:37:09.044279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:09.057441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:09.158278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1ed4e-7cf2-47e1-8848-081f62a1e277 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:09.195664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:09.196280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.196558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.196864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.197521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:09.199020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:09.201668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING keystone.server.flask.common [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 07:37:09.256400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.sql.core [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:37:09.256400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:37:09.256400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [parameters: {'id': '811c68ed56864b55a55479e8077d7288', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:37:09.256400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88234) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:37:09.267892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6011a3-fb34-48d1-989f-eca4e93e0967 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.268400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6011a3-fb34-48d1-989f-eca4e93e0967 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.268480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6011a3-fb34-48d1-989f-eca4e93e0967 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.270051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 474/947] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 811c68ed56864b55a55479e8077d7288.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 811c68ed56864b55a55479e8077d7288. Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:37:09.274616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result.read() Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 07:37:09.275426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application yield resource Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise self.value Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application session.commit() Aug 24 07:37:09.276796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:37:09.277698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:37:09.278601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result.read() Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:37:09.281066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [parameters: {'id': '811c68ed56864b55a55479e8077d7288', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.283272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:09.284587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 811c68ed56864b55a55479e8077d7288. Aug 24 07:37:09.287550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.288604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.288604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.288604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.292638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 474/948] 10.222.0.72 () {60 vars in 1140 bytes} [Wed Aug 24 07:37:09 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:09.295317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:09.297655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:09.298941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:09.373074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:09.396805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 475/949] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:09.423550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c133fab-8dc5-4b59-b9d2-fb3e9ea1aac4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:09.459596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:09.460559 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.461073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.461529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.468607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:09.471033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:09.482624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:09.483513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:09.484496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.491282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 475/950] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:09.492824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:09.574463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:09.597574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c099f-3ad2-4d32-b92a-bb8232afee6b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:09.634617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:09.635449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.635890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.636317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.637180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:09.637675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:09.640157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:09.659087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 476/951] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:09.714363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:09.839289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2d4c9-373e-4880-bd70-38440d0134db None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:09.867623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:09.877746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:09.882289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:09.882289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:09.882289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:09.882289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:09.882289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:09.890078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:09.892171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:09.893660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 476/952] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:09.900861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:10.031091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f36279-98b7-422a-818c-f47397212e6f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:10.068257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:10.068906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.069188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.069458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.070098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:10.070400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:10.071771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:10.077035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 477/953] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:10.085154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:10.192326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46aff2-6d3d-4262-b090-ef75427d2cd0 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:10.230637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:10.231639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.232116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.232589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.233956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:10.235247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:10.237432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c2cf7e-218c-4814-b691-7073805544c8 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:10.246160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 477/954] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:10.252718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:10.258582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:10.384741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e25a20-bbb3-45e4-97a2-c5c77160e9c4 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:10.433854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:10.434474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.434756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.435026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.435692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:10.437103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f64d9-4851-4c73-8fe5-b61f09cefcfb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:10.555523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 478/955] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:37:10.686695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8d70-5065-4eb8-9f1c-9966e9754acb None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.687479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8d70-5065-4eb8-9f1c-9966e9754acb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.687971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8d70-5065-4eb8-9f1c-9966e9754acb None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.689869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 478/956] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:10.695591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a1cb2-88b7-47a5-b018-7c005c877eb4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.695881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a1cb2-88b7-47a5-b018-7c005c877eb4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.696157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a1cb2-88b7-47a5-b018-7c005c877eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.715239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fb2a1cb2-88b7-47a5-b018-7c005c877eb4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:10.716477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fb2a1cb2-88b7-47a5-b018-7c005c877eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:10.778001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:10.799116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 479/957] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:10.805820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0081-bbec-40db-ae38-54b71d7aaf1b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.806024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0081-bbec-40db-ae38-54b71d7aaf1b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.806289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb0081-bbec-40db-ae38-54b71d7aaf1b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.807852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 479/958] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:10.813170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:10.813460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:10.813740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:10.833011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:10.834217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:10.909976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 480/959] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:11.090354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb0081-bbec-40db-ae38-54b71d7aaf1b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.154361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.184143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.210259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb0081-bbec-40db-ae38-54b71d7aaf1b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.250695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.251724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.252216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.252771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.253643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.255322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:11.265101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:11.266174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:11.266994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:11.269812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 480/960] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:11.281704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.305394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636effd-4f58-4c5f-8ade-761352279646 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.320507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.342483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.343357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.344351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.344801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.356544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:11.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:11.363339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:11.364404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:11.366783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 481/961] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:11.376572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.390550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cb10-d978-4105-a1eb-861f0caf3190 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.415559 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.439911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.440792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.441240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.441737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.443076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:11.443961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.445808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.452928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 481/962] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:11.458225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.511123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e945-3027-4f82-ac76-b4b91695c6b3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.548788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.550006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.550479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.550961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.551858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:11.552361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.554018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.567310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 482/963] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:11.568457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.574936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876652e-86b6-4399-86f0-43fe5714cf36 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.606132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.615124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] RBAC: Adding query filter params (type=compute) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.631283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189d94-91fe-4a9f-92b1-f1056d4fdd1a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.631821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 482/964] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:11.682268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae7b65-c84b-4f90-b0cb-5cdea12235ba None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.721542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.722184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.722457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.722723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.735644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.737263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:11.755204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:11.756491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:11.758112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:11.758112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:11.758112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:11.758288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e67f-4634-4b53-97a6-0b64a1e3b43c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.758288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e67f-4634-4b53-97a6-0b64a1e3b43c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.758288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e67f-4634-4b53-97a6-0b64a1e3b43c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.758502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 483/965] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:11.761763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 483/966] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:11.762602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:11.763037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.763234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.763514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.785483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:11.787022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:11.835082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.873249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:11.876194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 484/967] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:11.891492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3649f1-9cdf-456e-8b9d-18bfe8c27ce1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:11.927126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:11.927734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:11.928012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:11.928285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:11.928965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:11.929271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:11.930717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:11.944398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 484/968] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:11.949895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.055870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bacfda-1ccf-45e3-bd9f-a6bd8c636928 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.096883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.097655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.097849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.098127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.105600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.107065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:12.119006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:12.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.121066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 485/969] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:37:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:12.126757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.156026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.206629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:12.241135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6941c7-459a-4022-af00-a5141b231196 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.281201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.282204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.282676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.283167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.284336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:12.284907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.286632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.294469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c12eda-2532-4a8f-a68a-918c5e6cc5fc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.307041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 485/970] 10.222.0.72 () {56 vars in 1149 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:12.314703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.325057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:12.341098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.341844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.342186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.342557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.343260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.344950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.393134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.sql.core [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:37:12.393134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:37:12.393134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [parameters: {'user_id': 'd9fce90ff6b3451094c1036359d98f6a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:37:12.393134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88234) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result.read() Aug 24 07:37:12.399844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:37:12.400793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application yield resource Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application next(self.gen) Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise self.value Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application session.commit() Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 07:37:12.401761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.session.flush() Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application compat.raise_( Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 07:37:12.402668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 07:37:12.403569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application conn.query(q) Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application result.read() Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 07:37:12.405638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd9fce90ff6b3451094c1036359d98f6a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:12.407379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 07:37:12.409676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.414428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 486/971] 10.222.0.72 () {60 vars in 1131 bytes} [Wed Aug 24 07:37:12 2022] POST /identity/v3/users => generated 160 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:12.421483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.457726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493105f-57ed-4fbc-9e29-7d46ca9c8e68 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.507897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.509246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.509246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.509549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.544317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:12.549550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7b4ca-12a4-4519-93fd-d6cb25b00f36 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.588696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.590156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f8b74-5758-4780-baa1-604b33e876e0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.596451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.597139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.597419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.597695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.598886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:12.606455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.607795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-9915f5fe-ffb4-47ff-ae52-1dea91bfe7d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:12.619870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:12.620931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 07:37:12.622227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:12.624499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 487/972] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:12.626779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43810-87e5-4be4-a08e-09aded590fd6 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.627057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43810-87e5-4be4-a08e-09aded590fd6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.627327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43810-87e5-4be4-a08e-09aded590fd6 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.631509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 488/973] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:12.632237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43810-87e5-4be4-a08e-09aded590fd6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.708781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 486/974] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:12.708781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.708781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.708781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.724908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:12.726441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:12.749246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43810-87e5-4be4-a08e-09aded590fd6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.787554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.788260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.794724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.794724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.794724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:12.794724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.794724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.796007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 487/975] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:12.796807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:12.837025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:12.852491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 489/976] 10.222.0.72 () {56 vars in 1101 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:12.921017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a86a1d-b642-4de0-9de1-4cc2ea4f0d4d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:12.960170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:12.960765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:12.961046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:12.961317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:12.973112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:12.974586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:12.990598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:12.991595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:12.992525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:12.992525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:12.992525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:12.992525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 488/977] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:12.997692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.019526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.087463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.113630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f3375-d1fa-4ea8-b256-e140ce2245bf None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.159839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.160977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.161256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.161690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.162418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.162822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.164069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.176927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 490/978] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:13.182326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.200154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.214405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfebce-36c4-4a2a-a975-375965b077fa None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.254718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.255346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.255722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.256019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.256577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.258005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.267214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:13.267799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:13.268779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:13.269658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 489/979] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:13.274866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.294674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad9265-d832-4fe9-b02a-a528b2526303 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.333177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.334158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.334437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.334708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.341792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.343228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:13.356547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:13.357507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:13.358392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 491/980] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:13.363924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.383479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0744f9-745c-443d-8990-aa6c8a7da69b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.418603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.419210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.419488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.419753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.420387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.420744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.422657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.427834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 490/981] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:13.433467 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.453233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.483563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc732cbe-8a54-4b88-9d78-47997e19c226 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.519231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.520116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.520394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.520691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.521909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.522240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.523691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.526293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.536603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 492/982] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:13.542448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.547884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c869239-c880-46f5-83ba-a74bbe1b9440 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.583600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.584378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.584491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.584786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.585411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.585709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.587065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.597009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 491/983] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:13.598091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.655286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e1b7d-ca01-41cd-930f-da1de11814ee None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.682849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.691504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.692107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.701824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.701824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.701824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] RBAC: Adding query filter params (role.id=039cf3c7d1ae4cedab5ed8f9b894aaad, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=0c877a807efb45a8b6475e0c8db99e98) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.701824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.701824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ff405-e145-44ea-a2e0-f66c5c34db9a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.702480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 493/984] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/role_assignments?user.id=0c877a807efb45a8b6475e0c8db99e98&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=039cf3c7d1ae4cedab5ed8f9b894aaad => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:13.702771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd741-dd3f-4776-9487-a5b2125476a1 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.703059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd741-dd3f-4776-9487-a5b2125476a1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.703325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd741-dd3f-4776-9487-a5b2125476a1 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.704617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 494/985] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:13.710276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.710641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.710981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.715178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef181-ac09-4968-ae1d-a65e30478877 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.732174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:13.733864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:13.758458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.759093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.759374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.759672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.760404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] RBAC: Adding query filter params (interface=admin, service_id=2039940db07d47c9abe0297996276bd8, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:13.760766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.762149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.769147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 492/986] 10.222.0.72 () {56 vars in 1249 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/endpoints?service_id=2039940db07d47c9abe0297996276bd8&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:13.774548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:13.799422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:13.820801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 495/987] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:13.887851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7823-6b0e-4ba6-a005-0dcead9220af None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:13.923984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:13.924856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:13.925152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:13.925426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:13.926033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:13.927404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73cdcfc-e3d7-4d24-adbf-ca3910827869 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:13.933553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 493/988] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:13 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:37:14.088436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:14.194423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be73ff-f366-4b87-8d86-a1b244ba084d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:14.228996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:14.229611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:14.229884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:14.230150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:14.230793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:14.232157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bfb2e-d1c6-48fa-9344-8e91e9c3d5b0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:14.312683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 496/989] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:14 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:15.654171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ca62-2028-49a1-b11b-f82b481791e4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:15.654764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ca62-2028-49a1-b11b-f82b481791e4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:15.654764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69ca62-2028-49a1-b11b-f82b481791e4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:15.656221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 494/990] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:15.661240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c5a-2d0c-4f9c-b1f7-130bc29ae235 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:15.661789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c5a-2d0c-4f9c-b1f7-130bc29ae235 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:15.661789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e6c5a-2d0c-4f9c-b1f7-130bc29ae235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:15.680018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2b6e6c5a-2d0c-4f9c-b1f7-130bc29ae235 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:15.681241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2b6e6c5a-2d0c-4f9c-b1f7-130bc29ae235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:15.729022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:15.757341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 497/991] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:15.820833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e56d46-0ba3-40a7-b8fc-08d7b2ce9518 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:15.821141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e56d46-0ba3-40a7-b8fc-08d7b2ce9518 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:15.821379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e56d46-0ba3-40a7-b8fc-08d7b2ce9518 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:15.822832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 495/992] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:15.828532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:15.828850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:15.829122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:15.847193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:15.848365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:15.915356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 498/993] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:16.024999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e56d46-0ba3-40a7-b8fc-08d7b2ce9518 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.036326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.128792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e56d46-0ba3-40a7-b8fc-08d7b2ce9518 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.161629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.162262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.162540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.162811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] PATH_INFO: `/v3/services/image` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.163392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.164785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:16.174049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:16.175265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:16.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.181850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 496/994] 10.222.0.72 () {56 vars in 1105 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:16.181973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.182055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.254209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.282606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60523d52-7b7b-4360-9ad1-1e70dfe21cb3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.283608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb65116-4bf5-4d35-90e7-83ef5a89689c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.291005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.318064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.318305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.318678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.318777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.318956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.319049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.319226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.319320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.319858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] RBAC: Adding query filter params (name=image) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:16.319948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.320159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.321451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.321547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5d47b-53b0-4869-90ce-cf80fc46cde0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.326310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 499/995] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:16.327556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c501a-5894-4873-baed-c10fad106d9f None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.327836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c501a-5894-4873-baed-c10fad106d9f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.328109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c501a-5894-4873-baed-c10fad106d9f None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:16.330335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:16.331309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.332236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 497/996] 10.222.0.72 () {56 vars in 1126 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:16.333111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.333935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 500/997] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:16.335133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.335410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.335681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.353219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:16.354491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:16.424193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 501/998] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:16.425274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.442645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad995afe-efbd-4442-bc1b-0c070a7faeda None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.486356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.487517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.487812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.488285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.489695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] RBAC: Adding query filter params (type=image) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:16.489954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.492425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.495725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.512984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 498/999] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:16.553389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.556546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c371e6c-21f9-41aa-86f8-ba9ff85471d2 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.594628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.595244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.595519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.595787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.596427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:16.596758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.598137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.608638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 502/1000] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:16.610911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.723368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9d882-2b9f-4443-8195-0d12fa91415d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.764276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.765732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.766164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.766583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.767724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:16.768442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.771011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.779324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 499/1001] 10.222.0.72 () {56 vars in 1122 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:16.785277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.817699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:16.884445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:16.895866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f281f-05f7-485c-b328-bfe767d5ffc6 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.927371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a1c4f-09ba-4592-b32b-e416eb43c638 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:16.932726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.933337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.933612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.933882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.934561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.935922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.961783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:16.962413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:16.962688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:16.962986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:16.963564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:16.964951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:16.974343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:16.975884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:16.977041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 500/1002] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:16.981182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:17.052690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 503/1003] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:17.096897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fdecb-135c-44f4-9fee-57762e157c33 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:17.129676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:17.146219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:17.146887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:17.147166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:17.147431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:17.148120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:17.148421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:17.149839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:17.159966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 501/1004] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:17.161518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:17.190340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:17.269853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97076-aee2-46c3-ba41-b5732464fa9c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:17.304317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:17.304955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:17.305228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:17.305494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:17.306121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:17.306419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:17.307773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:17.318089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 504/1005] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:17.319349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:17.429454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4771-00f6-4ab3-a2a0-eb637e2a0566 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:17.464800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:17.465405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:17.465681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:17.465947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:17.466633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] RBAC: Adding query filter params (interface=public, service_id=3d55a087ec064af09f1996a6ec29ae9e, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:17.466936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:17.468319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4d7a6-c85c-45ac-b984-160c97805aa1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:17.478040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 502/1006] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:37:17 2022] GET /identity/v3/endpoints?service_id=3d55a087ec064af09f1996a6ec29ae9e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:17.479297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:17.574771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:17.586933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a61c-ff7b-4856-9aad-74d76d490ed9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:17.621060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:17.621667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:17.621940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:17.622206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:17.622806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:17.624151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f3666-35b7-4a52-9118-d7e0a91b5c3e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:17.631195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 505/1007] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:17 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:37:18.506309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684827f3-3fa3-4c5d-ba5a-28b35c6a575e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:18.506860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684827f3-3fa3-4c5d-ba5a-28b35c6a575e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:18.506860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684827f3-3fa3-4c5d-ba5a-28b35c6a575e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:18.508356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 503/1008] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:18.513557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c862-ecfc-4176-913e-303dcf91de16 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:18.513936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c862-ecfc-4176-913e-303dcf91de16 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:18.514109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2c862-ecfc-4176-913e-303dcf91de16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:18.533847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8db2c862-ecfc-4176-913e-303dcf91de16 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:18.535041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8db2c862-ecfc-4176-913e-303dcf91de16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:18.603442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 506/1009] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:18.873123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684827f3-3fa3-4c5d-ba5a-28b35c6a575e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:18.931379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:18.980395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684827f3-3fa3-4c5d-ba5a-28b35c6a575e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:18.991223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca67c-d5c5-4672-b426-4d0849ddfb6c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:18.991515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca67c-d5c5-4672-b426-4d0849ddfb6c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:18.991789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ca67c-d5c5-4672-b426-4d0849ddfb6c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:18.993427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 507/1010] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:18.998398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:18.998679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:18.998951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.018054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:19.018658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:19.018936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:19.019206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.019363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:19.019887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:19.020513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:19.021390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:19.089323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 508/1011] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:19.131514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 504/1012] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:18 2022] POST /identity/v3/services => generated 242 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:19.359605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:19.375042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:19.472307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0ede5-f971-4cb8-80f5-d2461c4900af None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:19.512896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:19.513532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:19.513806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:19.514331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.515711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:19.517159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145b44-756e-42df-9ec4-0cc9a1e2a022 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:19.518912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 509/1013] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:19.524476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:19.583231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8cc24-5c0b-4062-a27f-1c7dd20425c9 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:19.583471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8cc24-5c0b-4062-a27f-1c7dd20425c9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:19.583622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8cc24-5c0b-4062-a27f-1c7dd20425c9 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.586094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 510/1014] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:19.590366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:19.590642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:19.590911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.609279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:19.610462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:19.634695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda209e-974f-4d2e-8c10-d7ebdd84616a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:19.649648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:19.671841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:19.672498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:19.672810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:19.673199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:19.673916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:19.675800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:19.680755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 511/1015] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:19.769031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 505/1016] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:19 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:37:19.948258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:19.956580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:20.051737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af25a6c-33f6-4d6e-9452-257e93c65c1b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:20.087367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:20.088447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:20.088966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:20.089457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:20.090523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:20.092857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:20.105243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:20.106260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:20.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 512/1017] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:20.114561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:20.216976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630067-f34a-4999-a4e3-4e40043ce088 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:20.254436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:20.255053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:20.255330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:20.255600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:20.256236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] RBAC: Adding query filter params (name=compute) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:20.256536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:20.257948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:20.267552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 506/1018] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:20.268871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:20.340458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:20.372263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b9b3a-c87b-4faf-b6f6-d4b61ed78385 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:20.414386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:20.414999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:20.415278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:20.415551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:20.416179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:20.416484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:20.417942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f38ca-c8f4-4638-872f-2c351235a321 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:20.428077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 513/1019] 10.222.0.72 () {56 vars in 1112 bytes} [Wed Aug 24 07:37:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:20.429371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:20.460564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:20.536369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b8e63-999b-4dad-8ff2-2909aa13695b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:20.573459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:20.574332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:20.574809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:20.575258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:20.576087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:20.578229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006787d7-b117-434e-8e2c-f83a2c65a2b7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:20.662789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 507/1020] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:21.084337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd49ae-fb8e-460c-aaf0-77b1e671f579 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.084886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd49ae-fb8e-460c-aaf0-77b1e671f579 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.084989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd49ae-fb8e-460c-aaf0-77b1e671f579 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.086292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 514/1021] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:21.091506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.091800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.092077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.110382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:21.111578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:21.189953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 508/1022] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:21.496087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd49ae-fb8e-460c-aaf0-77b1e671f579 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:21.602681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd49ae-fb8e-460c-aaf0-77b1e671f579 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:21.637834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:21.638906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.639381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.640584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.641393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:21.642962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:21.644988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-6fba0287-40a2-42ff-a608-f3e0dd9888c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:21.652669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:21.662971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:21.663836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 515/1023] 10.222.0.72 () {56 vars in 1105 bytes} [Wed Aug 24 07:37:21 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:21.664697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:21.673514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:21.763520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ead868-78c1-417d-9a5a-2e5c98da759a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:21.769528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719e7e-40d4-4dba-864c-5aad2dbe768b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.769820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719e7e-40d4-4dba-864c-5aad2dbe768b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.770104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34719e7e-40d4-4dba-864c-5aad2dbe768b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.771611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 516/1024] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:21.776763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.777048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.777321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.795787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:21.798532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:21.813186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:21.813818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.814114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.814355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.814990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] RBAC: Adding query filter params (name=image) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:21.815292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:21.816759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:21.822414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 509/1025] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:21.828494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:21.879121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 517/1026] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:21.908104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:21.932946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee9044-3d4f-4499-ad98-4dcd52293f01 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:21.968227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:21.969154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:21.969691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:21.970156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:21.971008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:21.971488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:21.973098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:21.978453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 510/1027] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:21.984123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:22.086858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e0a4e-61ee-467e-9233-b7be17592111 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:22.116886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:22.126665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] RBAC: Adding query filter params (interface=public, service_id=479015ea005e4af2bf5098603471ca0f, region_id=RegionOne) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:22.132608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:22.139583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 518/1028] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:37:21 2022] GET /identity/v3/endpoints?service_id=479015ea005e4af2bf5098603471ca0f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:22.142049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:22.152800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:22.259810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d63cb-ca31-43d3-a973-02dd12c0a584 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:22.261022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de8745-9091-46b6-aa46-6f9c87cf99f3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:22.295235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:22.296364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.296637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.296929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.297512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:22.298030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:22.298131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.298383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.298653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.299563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:22.299769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:22.300945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4b0f4-23d3-4985-9aee-3c35610626c5 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:22.301591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 519/1029] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:22.306978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:22.308011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 511/1030] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:37:22 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:37:22.386534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:22.433714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696869b0-df66-4d0b-88ce-1c2bd913dee4 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:22.471989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:22.472605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.472916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.473178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.473820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:22.475244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb40eb9-7648-46b8-b18b-b1f49bb0e8b2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:22.621610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 520/1031] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:37:22 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:37:22.762939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae362-d675-4241-b2cb-e5882e5b9a4b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.763472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae362-d675-4241-b2cb-e5882e5b9a4b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.763472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae362-d675-4241-b2cb-e5882e5b9a4b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.765521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 512/1032] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:22.770678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:22.770968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:22.771251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:22.790951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:22.792154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:22.865082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 521/1033] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:23.134842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae362-d675-4241-b2cb-e5882e5b9a4b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:23.136276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:23.236831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae362-d675-4241-b2cb-e5882e5b9a4b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:23.271386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:23.272009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:23.272288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:23.272560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:23.282867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:23.287636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:23.288595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:23.289772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:23.290853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 513/1034] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:23.294825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:23.345019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:23.402592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518c702-9af7-46e4-b1e7-57de8e49780c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:23.440003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:23.440622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:23.440928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:23.441200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:23.441867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:23.442170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:23.443541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:23.452138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 522/1035] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:23.457763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:23.521954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:23.568532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e81b6-4eaa-411a-ade0-b1dfb0f5164c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:23.605303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:23.605924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:23.606202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:23.606472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:23.616493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:23.617962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:23.632613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:23.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:23.634528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:23.634528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:23.634528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:23.635271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 514/1036] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:23.640135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:23.743907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcb389-dcde-45ae-bb81-98592a77c5f0 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:23.752887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:23.780716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:23.781332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:23.781607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:23.781876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:23.782523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:23.782825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:23.784177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:23.800719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 523/1037] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:23.802589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:23.907748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80437f5-00de-42b4-9f37-6475cfdacab1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:23.921724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:23.943038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567741ef-ac37-4872-860d-24359db6988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:23.943673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567741ef-ac37-4872-860d-24359db6988b None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:23.943956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567741ef-ac37-4872-860d-24359db6988b None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:23.944227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567741ef-ac37-4872-860d-24359db6988b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:23.952534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567741ef-ac37-4872-860d-24359db6988b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:23.954005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567741ef-ac37-4872-860d-24359db6988b None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-567741ef-ac37-4872-860d-24359db6988b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:23.965892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:23.966883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:23.972889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 515/1038] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:23.973576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.078074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a0683-8c6e-46f6-b941-b7d8bdbd55df None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.112447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:24.113355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.113787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.114210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.115106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:24.115563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:24.117179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e43e345-15ad-41da-bee3-69b1b81e371f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:24.135494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 524/1039] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:37:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:24.141887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567741ef-ac37-4872-860d-24359db6988b None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.254458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab650-066b-4c16-8403-e94d1f4bf3cf None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.254785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab650-066b-4c16-8403-e94d1f4bf3cf None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.255006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab650-066b-4c16-8403-e94d1f4bf3cf None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.256568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 525/1040] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:24.261645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.261925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.262198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.277729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567741ef-ac37-4872-860d-24359db6988b None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.280923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:24.282093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:24.303283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.317983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:24.318589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.319025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.319298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.352202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 526/1041] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:24.369951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.398482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:24.402530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e7ef7-7261-4e1d-bba0-6bef3a632f5f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:24.512411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 516/1042] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:37:24.514134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.614156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ffb9-4c8d-4936-bff8-0551dabb3544 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.614456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ffb9-4c8d-4936-bff8-0551dabb3544 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.614631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ffb9-4c8d-4936-bff8-0551dabb3544 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.616220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 517/1043] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:24.621090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.621284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8df8c-6a46-475d-a185-13e61c2dd8e6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.621388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.621624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.640588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:24.641799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:24.656903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:24.657533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.657956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.658228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.668051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:24.669516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:24.683911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:24.684894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:24.685981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:24.685981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:24.685981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:24.685981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 527/1044] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:24.687755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.698642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.708346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 518/1045] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:24.709215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.712227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.793607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd8432-f49b-4078-a9bf-cfcc5e94a2a9 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.819724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd750f-5e71-4090-9967-41f269b54e71 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.831018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:24.831905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.832331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.832781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] PATH_INFO: `/v3/services/image` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.833688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:24.835441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:24.845152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:24.846136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:24.846977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 528/1046] 10.222.0.72 () {56 vars in 1105 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:24.852254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.859913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:24.860526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:24.860847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:24.861121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:24.861770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:24.862493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:24.863874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:24.877701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 519/1047] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:24.883435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:24.926758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.948901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:24.960915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd6462-9d79-4042-9163-be98d64f5c8a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:24.987029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98c31-be94-4a39-b0be-533d7d54328f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.000238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.000883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.001167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.001441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.002077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] RBAC: Adding query filter params (name=image) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:25.002380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.003778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.008717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 529/1048] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.009518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.022855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.023478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.023751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.024016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.030791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.032216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:25.043136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:25.044036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:25.044956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:25.049888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 520/1049] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:25.050709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.113622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367405-13eb-43d3-9985-3c91c6274fde None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.147442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.147985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.148259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.148526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.150486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4734cf-60f7-487b-8136-b73e710b9eb3 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.151124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.159224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:25.163973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:25.164993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:25.165843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 530/1050] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:25.166624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.178307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.189268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.189874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.190153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.190425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.191061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] RBAC: Adding query filter params (type=image) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:25.191355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.192854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.197622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 521/1051] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.198396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.270156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912f934-ea36-4957-8944-33e201121a63 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.297878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632a3cf-11f3-431b-83fa-55cd30f5c311 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.303966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.304567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.304878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.305157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.305846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:25.306178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.307601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.320044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 531/1052] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.320851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.336811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.337416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.337692 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.337962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.338616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:25.338916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.340286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.348940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 522/1053] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.349719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.379697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.400580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.425074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ec3bc-7bd0-4f64-85c4-e3c0b2b4396c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.452067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3f55c-9294-4302-a6a2-b94c36b7e6f1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.460343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.460977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.461250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.461515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.462197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.463554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.487409 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.488010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.488289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.488570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.489323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] RBAC: Adding query filter params (role.id=039cf3c7d1ae4cedab5ed8f9b894aaad, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:25.489634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.491052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.496356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 523/1054] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=039cf3c7d1ae4cedab5ed8f9b894aaad => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.497164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.586016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 532/1055] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:37:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:25.634436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54604dad-e467-412d-9586-b5e7f4dd2504 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.651599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.682437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.683615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.683896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.684168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] PATH_INFO: `/v3/users/a1c278b99ab042a8b0a4619e8e5f2041` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.699322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] RBAC: Authorizing `identity:get_user(user_id=a1c278b99ab042a8b0a4619e8e5f2041)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.700826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:25.702630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 524/1056] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/users/a1c278b99ab042a8b0a4619e8e5f2041 => generated 285 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:25.709308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:25.824645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5bcd0-a8bf-453d-a077-4035cffc2992 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:25.839585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:25.861348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:25.862020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:25.862303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:25.862577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:25.951351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:25.953246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:26.056788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 533/1057] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:26.060770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:26.164455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8d9b2-b4d7-4b0a-a656-3c32a622873a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:26.198976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:26.199604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:26.199878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:26.200144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:26.201287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:26.203324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e3922-9c42-4169-a12a-e9364fbd55e2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:26.208197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 525/1058] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:26 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:26.211467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:26.226199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:26.317088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1848e0-7e70-4d07-a796-f180a15c5068 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:26.351851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:26.352457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:26.352762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:26.353037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:26.353764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=b58a5ff1900746f9949ae1b474a5c965, user.id=a1c278b99ab042a8b0a4619e8e5f2041) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:26.354072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:26.355506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2949a-e5c6-4cad-9864-937f1de98607 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:26.364792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 534/1059] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:26 2022] GET /identity/v3/role_assignments?user.id=a1c278b99ab042a8b0a4619e8e5f2041&scope.project.id=b58a5ff1900746f9949ae1b474a5c965&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:27.390276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b24ad-4a43-492a-9f2d-8101404f5da2 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:27.390807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b24ad-4a43-492a-9f2d-8101404f5da2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:27.390807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b24ad-4a43-492a-9f2d-8101404f5da2 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:27.392469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 526/1060] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:27.397527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ff52-dd1b-4c95-8a68-fc597acdac7c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:27.397928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ff52-dd1b-4c95-8a68-fc597acdac7c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:27.398115 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108ff52-dd1b-4c95-8a68-fc597acdac7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:27.420633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-1108ff52-dd1b-4c95-8a68-fc597acdac7c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:27.421969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1108ff52-dd1b-4c95-8a68-fc597acdac7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:27.491446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 535/1061] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:27.531162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b57e-5d5f-4c1e-86d8-b1d73fe58c48 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:27.531498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b57e-5d5f-4c1e-86d8-b1d73fe58c48 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:27.531720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3b57e-5d5f-4c1e-86d8-b1d73fe58c48 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:27.533106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 527/1062] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:27.538256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:27.538541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:27.538810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:27.551407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:27.558070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:27.559252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:27.627228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 536/1063] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:27.757670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b57e-5d5f-4c1e-86d8-b1d73fe58c48 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:27.794078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:27.862302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3b57e-5d5f-4c1e-86d8-b1d73fe58c48 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:27.892129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:27.899835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:27.900572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:27.900869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:27.901137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:27.912127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:27.917152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:27.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:27.919394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 528/1064] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:27.924872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.005342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a275-9623-43db-9dc9-ddeba89422fc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.024971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.031093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b40731-e806-4428-9258-1e6ca9e6ebde None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.044085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.044721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.044997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.045866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.050411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.066590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.067352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.067711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.067994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.068677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:28.069002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.070374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.080689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 529/1065] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:28.085241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.122402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.124919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.190202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b5f19-02e4-4e72-ad32-6c873ee3363a None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.226420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.226978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.227256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.227526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.238378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.239805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 537/1066] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:37:28.239929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.245524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c93a2dff-906f-43fb-b0a4-80a3303b78fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:28.256069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:28.257076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:28.258025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:28.258025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:28.258025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:28.258025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 530/1067] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:28.258698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8d0c-48db-4fc4-b65b-e0a13c1495bb None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.258698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8d0c-48db-4fc4-b65b-e0a13c1495bb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.259009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c8d0c-48db-4fc4-b65b-e0a13c1495bb None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.260238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 531/1068] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:28.264753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8d0c-48db-4fc4-b65b-e0a13c1495bb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.277407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.282223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.357060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf126-caee-4421-8277-5074d8de7c80 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.375550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c8d0c-48db-4fc4-b65b-e0a13c1495bb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.393998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.394622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.394896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.395158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.406811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.408272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.414116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.414723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.415004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.415274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.415914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:28.416216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.417593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-c1361d50-2057-4eeb-9dd3-c2ea95e6c795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:28.423305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:28.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 538/1069] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.430998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.431511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 532/1070] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:28.431511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.446534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:28.447740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:28.501124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.515998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 539/1071] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:28.516867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.533217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5883-67c7-4c08-8dfc-a364c4feef26 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.535601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.583930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.584595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.584910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.585180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.585815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:28.586113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.587473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.600535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 533/1072] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:28.638781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3f65d-c034-4832-80da-198e8809a4a2 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.678056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.678962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.679393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.679814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.688349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.690117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:28.703276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:28.704379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:28.705367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:28.705927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 540/1073] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:28.711087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.789845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.827637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed7d69-7028-4cf1-9062-41966d20f1f4 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.863155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.870152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.870826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.871107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.871376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.872148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:28.872459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:28.873954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:28.887112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:28.893025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 534/1074] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:28.899400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:28.911366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05456-3e30-46cc-8a51-40cf1b6e770b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:28.945493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:28.946123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:28.946402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:28.946674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:28.958308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:28.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:28.964020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:28.964887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 541/1075] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:28.970974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.002156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf0792-67d1-42d8-93d5-62f4d6dd1f43 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.038164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.038785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.039052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.039312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.049654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.051080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:29.064981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:29.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:29.067140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:29.067140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:29.067140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:29.067140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 535/1076] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:29.072003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.072425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9288c0-17fb-4c42-b052-aaf55bca5a09 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.106985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.107611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.107887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.108157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.108841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:29.109141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.110516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.113894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.115512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.119311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 542/1077] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:29.125859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.171998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a992-97a9-4bd1-83e1-461af3a35bc9 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.207582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2097-ded6-4839-8def-08b48295751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.208181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2097-ded6-4839-8def-08b48295751e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.208455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2097-ded6-4839-8def-08b48295751e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.208753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2097-ded6-4839-8def-08b48295751e None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.209395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2097-ded6-4839-8def-08b48295751e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:29.209700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2097-ded6-4839-8def-08b48295751e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.211061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2097-ded6-4839-8def-08b48295751e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.222619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 536/1078] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:29.224928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9df9e-1b39-4b72-b128-a8f0fa63455d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.228760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2097-ded6-4839-8def-08b48295751e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.258459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.259077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.259351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.259618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] PATH_INFO: `/v3/users/a1c278b99ab042a8b0a4619e8e5f2041` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.260782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] RBAC: Authorizing `identity:get_user(user_id=a1c278b99ab042a8b0a4619e8e5f2041)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.262247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.264601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 543/1079] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:29 2022] GET /identity/v3/users/a1c278b99ab042a8b0a4619e8e5f2041 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:29.270024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.325212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.328570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2097-ded6-4839-8def-08b48295751e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.334948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.360997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.361623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.361903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.362168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.366959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b1e96-10a7-47e9-9718-f9f21c1d822e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.369211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.371028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:29.382833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:29.383839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:29.384949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 537/1080] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:29.390338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.400986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.401618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.401891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.402153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.403495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.405771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.407438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 544/1081] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:29 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:29.413141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.493991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9837eb-a80f-4d6d-9c31-276bb6b05746 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.510141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d77270-6e01-47b8-bcc6-b7e3e55ea36b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.528910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.529543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.529818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.530084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.530768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:29.531068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.532489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.543830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 538/1082] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:29.546316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.546977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.547257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.547532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965/users/a1c278b99ab042a8b0a4619e8e5f2041/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.549826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] RBAC: Authorizing `identity:create_grant(project_id=b58a5ff1900746f9949ae1b474a5c965, user_id=a1c278b99ab042a8b0a4619e8e5f2041, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.550484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:29.551251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb90a8-049a-4a6d-a751-6507b543f4ea None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.554380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.572236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:29.650339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a3ab-96df-4e7b-936a-e26ed87e5605 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:29.683090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:29.683729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:29.684005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:29.684274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/039cf3c7d1ae4cedab5ed8f9b894aaad` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:29.686995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=d882cbdd492344c7a3638c65ed08b32d, role_id=039cf3c7d1ae4cedab5ed8f9b894aaad)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:29.688498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160db3d-0053-4f84-8887-04c11ddfcddd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:29.694813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 545/1083] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:37:29 2022] PUT /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965/users/a1c278b99ab042a8b0a4619e8e5f2041/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:37:29.764755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 539/1084] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:37:29 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/d882cbdd492344c7a3638c65ed08b32d/roles/039cf3c7d1ae4cedab5ed8f9b894aaad => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:37:31.576800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909b60-b6cc-46cd-92bd-e6a7c840a512 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:31.577361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909b60-b6cc-46cd-92bd-e6a7c840a512 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:31.577361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909b60-b6cc-46cd-92bd-e6a7c840a512 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:31.578689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 546/1085] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:31.584041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:31.584475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:31.584689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:31.602587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:31.603786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:31.649577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff45e74-829d-429a-8c1f-79e9bf7cf5f2 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:31.649861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff45e74-829d-429a-8c1f-79e9bf7cf5f2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:31.650145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff45e74-829d-429a-8c1f-79e9bf7cf5f2 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:31.651479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 547/1086] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:37:31.656555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:31.656810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:31.657087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:31.671266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 540/1087] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:31.674986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:31.680586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:31.746856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 548/1088] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:37:31.939295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:31.950968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.017393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.038458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19f9bc-0004-4cbb-9bb2-6edc7b48f1a9 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.074103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c686d-46b0-471f-9317-69409aab225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.074725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c686d-46b0-471f-9317-69409aab225d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.075061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c686d-46b0-471f-9317-69409aab225d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.075337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c686d-46b0-471f-9317-69409aab225d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.085500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c686d-46b0-471f-9317-69409aab225d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-467c686d-46b0-471f-9317-69409aab225d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:32.090229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:32.094217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:32.095115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 541/1089] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:32.097497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c686d-46b0-471f-9317-69409aab225d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.118235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac6c7b-f597-4b64-9bd8-aafd2e908a3e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.144476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.152847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.153469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.153741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.154009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.164213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:32.167716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:32.169277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:32.170852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:32.171697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 549/1090] 10.222.0.72 () {56 vars in 1120 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:32.177196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.198892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c686d-46b0-471f-9317-69409aab225d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.232094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.233295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.233570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.233843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.234507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:32.234810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.236176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.244834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 542/1091] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:32.250360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.278097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f6d62-beed-4678-b8de-3a860c57c0ad None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.312380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.313000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.313278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.313541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.314199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:32.314494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.315844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.329021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 550/1092] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:32.330205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.347080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92aed22-27be-43d5-b744-afbc4c2ad067 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.381801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.383050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.387086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.387728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.388094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.388404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] PATH_INFO: `/v3/users/a1c278b99ab042a8b0a4619e8e5f2041` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.389538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] RBAC: Authorizing `identity:get_user(user_id=a1c278b99ab042a8b0a4619e8e5f2041)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.393265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.393265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 543/1093] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/users/a1c278b99ab042a8b0a4619e8e5f2041 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:32.398377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.436369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6783120-0e41-42a0-891c-41f59b356063 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.470566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.471122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.471391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.471662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.481840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.483304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:32.497162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:32.498670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:32.499568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:32.499568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:32.499568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:32.499568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 551/1094] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:32.501264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc76747-b683-4539-a42b-7d480a289f58 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.504544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.536015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.536618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.536930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.537200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.602449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.605521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b426f-4cb4-4a13-a8d5-2cae2b95ac75 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.611764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.614105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.615563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.640972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.641580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.641854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.642123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.642759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:32.643117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.644461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.655570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 552/1095] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:32.661238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.716637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 544/1096] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:37:32.726402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.777225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3068-f227-4cc1-8b03-b88e74333450 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.812259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.812910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.813190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.813457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.820795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.822262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.823703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.826495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ea03b-1743-456d-9ebb-3929819312ee None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:37:32.833387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:32.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:32.835466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 553/1097] 10.222.0.72 () {56 vars in 1093 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:32.840286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.861556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.862018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:32.862180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.862453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.862725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.863861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.865328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.867089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 545/1098] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:32.873542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:32.939776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9f5c-300d-4352-bcd1-4681750e9d23 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.972491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:32.973186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:32.973332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6620cef6-d155-4a74-b402-bfb0d0d5c5fd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:32.973466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:32.973735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:32.974448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:32.974747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:32.976186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:32.993332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 554/1099] 10.222.0.72 () {56 vars in 1133 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:32.998979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:33.009343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.009947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.010223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.010493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.011220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=b58a5ff1900746f9949ae1b474a5c965, user.id=a1c278b99ab042a8b0a4619e8e5f2041) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:33.011528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.012979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.018841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 546/1100] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/role_assignments?user.id=a1c278b99ab042a8b0a4619e8e5f2041&scope.project.id=b58a5ff1900746f9949ae1b474a5c965&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:37:33.085743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:33.110219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2319c7-9b3d-4eb4-9926-91aff14daa1a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:33.146901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.147553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.147832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.148101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.217680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.219122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.293887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:33.315505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 555/1101] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:37:33.321308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:33.354129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:33.422838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c012cd-c9e9-4f21-85b7-0bae0fe847ff None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:33.456506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.457224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.457498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.457764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.469325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.470761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:37:33.484495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:33.485524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:37:33.486414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:37:33.486414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:37:33.486414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:33.486414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 547/1102] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:33.491520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:33.589414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29616c-8f81-46f0-978b-23ed68c0f2a7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:33.623955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.624565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.624831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.625096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.625726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:33.626020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.627458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.638226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 556/1103] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:33.644526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:33.728398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:33.743106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664b17a-dd7d-4bea-bde0-be75a7f3cba2 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:33.776262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.776893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.777166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.777428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.783777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.785296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:37:33.796287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:33.798499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:33.799667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 548/1104] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:37:33.803687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:33.843268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:33.901629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4f360-7397-42e2-baef-430441b361ac None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:33.933987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:33.934583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:33.934855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:33.935120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:33.936336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:33.936633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:33.938070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a95c4ff-236f-4470-8ec7-710c9950ba6f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:33.949586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 557/1105] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:37:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:33.956274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:34.063913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efbf24-6b01-4b49-93a0-9e3a6ffc1088 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:34.096613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:34.097163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:34.097442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:34.097712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:34.098433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] RBAC: Adding query filter params (role.id=039cf3c7d1ae4cedab5ed8f9b894aaad, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=d882cbdd492344c7a3638c65ed08b32d) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:34.098737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:34.100175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bc3ee-0712-4dfb-a140-27bf4a3538ef None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:34.105982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 549/1106] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:33 2022] GET /identity/v3/role_assignments?user.id=d882cbdd492344c7a3638c65ed08b32d&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=039cf3c7d1ae4cedab5ed8f9b894aaad => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:35.006520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf7f08-9e20-404a-8b0c-23a4bffe0ca8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:35.007055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf7f08-9e20-404a-8b0c-23a4bffe0ca8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:35.007055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf7f08-9e20-404a-8b0c-23a4bffe0ca8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:35.008385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 558/1107] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:35.014027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:35.014469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:35.014565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:35.032895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:35.034073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:35.035980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:35.100031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 550/1108] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:35.366779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf7f08-9e20-404a-8b0c-23a4bffe0ca8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:35.464936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:35.470296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf7f08-9e20-404a-8b0c-23a4bffe0ca8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:35.503094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:35.503720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:35.504028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:35.504306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:35.505722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:35.507538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92dd1c-7f7c-46ea-a363-50620b554644 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:35.509130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 559/1109] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:35.515710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:35.613169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae62ec5-d074-43be-b884-f889d12fbefe None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:35.646869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:35.647395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:35.647673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:35.647934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:35.648580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:35.650027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602373a-89d9-4ca2-8aa9-4b3b56b4a391 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:35.659609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:35.745939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 551/1110] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:37:35 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:37.714634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e57f-e6db-416d-9e33-72d3522d2c91 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:37.715140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e57f-e6db-416d-9e33-72d3522d2c91 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:37.715140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e57f-e6db-416d-9e33-72d3522d2c91 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:37.716985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 560/1111] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:37.722148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:37.722566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:37.722691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:37.741609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:37.742795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:37.813236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 552/1112] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:38.079945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e57f-e6db-416d-9e33-72d3522d2c91 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:38.185379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e57f-e6db-416d-9e33-72d3522d2c91 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:38.218081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:38.218699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:38.218975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:38.219245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] PATH_INFO: `/v3/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:38.220616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] RBAC: Authorizing `identity:get_role(role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:38.222004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:38.223309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 561/1113] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:38 2022] GET /identity/v3/roles/0bb62a6e624c40b4832f54f88761160b => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:38.229377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:38.325124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449a4ca-4060-4ddc-9c78-a2666c144ae5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:38.347729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:38.357865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:38.360790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:38.360790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:38.360790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] PATH_INFO: `/v3/users/1b203ba7a15a4e1486d38598da3e541b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:38.371370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] RBAC: Authorizing `identity:get_user(user_id=1b203ba7a15a4e1486d38598da3e541b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:38.373362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:38.374921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 553/1114] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:38 2022] GET /identity/v3/users/1b203ba7a15a4e1486d38598da3e541b => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:38.380911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:38.418142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:38.482175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331a60c-7f89-4da9-a009-0eff3ba44dda None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:38.515851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:38.516452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:38.516761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:38.517036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:38.581561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:38.583003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:38.614220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:38.684592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 562/1115] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:38.691262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:38.789429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe9c99-7dfd-4200-a218-c75134f40b63 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:38.822931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:38.823544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:38.823816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:38.824083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:38.825200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:38.826671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25c34-4c64-4c15-b50c-05f743ab9cd9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:38.828255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 554/1116] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:38 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:38.834515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:38.930841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cea9-640f-43e4-ac98-fafaba930253 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:38.962792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:38.963385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:38.963666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:38.963935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:38.964673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=b58a5ff1900746f9949ae1b474a5c965, user.id=1b203ba7a15a4e1486d38598da3e541b) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:38.964982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:38.966930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8620f4e-a9cd-4151-86d5-4f26ffe3f214 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:38.968642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:38.972573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 563/1117] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:38 2022] GET /identity/v3/role_assignments?user.id=1b203ba7a15a4e1486d38598da3e541b&scope.project.id=b58a5ff1900746f9949ae1b474a5c965&role.id=0bb62a6e624c40b4832f54f88761160b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:40.888181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb2d41-87f6-4d17-9cff-07bb16bf6fe5 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:40.888768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb2d41-87f6-4d17-9cff-07bb16bf6fe5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:40.888768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb2d41-87f6-4d17-9cff-07bb16bf6fe5 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:40.890103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 555/1118] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:40.895360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:40.895762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:40.895901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:40.913692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:40.914873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:40.979588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 564/1119] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:41.251971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb2d41-87f6-4d17-9cff-07bb16bf6fe5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:41.302652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:41.351734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb2d41-87f6-4d17-9cff-07bb16bf6fe5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:41.385309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:41.385932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:41.386209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:41.386479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] PATH_INFO: `/v3/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:41.387864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] RBAC: Authorizing `identity:get_role(role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:41.389281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:41.390596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 556/1120] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:41 2022] GET /identity/v3/roles/0bb62a6e624c40b4832f54f88761160b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:41.396360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:41.496324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd8b36-371b-4f5d-9852-9819bb6936eb None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:41.522370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:41.530408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:41.532787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:41.532787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:41.532787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] PATH_INFO: `/v3/users/1b203ba7a15a4e1486d38598da3e541b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:41.532787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] RBAC: Authorizing `identity:get_user(user_id=1b203ba7a15a4e1486d38598da3e541b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:41.534189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:41.535849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 565/1121] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:41 2022] GET /identity/v3/users/1b203ba7a15a4e1486d38598da3e541b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:41.541632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:41.638400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459512f-d471-4038-b6d3-966d0e799853 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:41.655478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:41.671393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601cb939-c869-4a7a-992a-9ae36d5776fa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:41.681701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 557/1122] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:41 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:41.682978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:41.782899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a376e9-ac94-4c5d-bef4-ed9e5b3faaef None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:41.815589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:41.816232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:41.816509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:41.816811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965/users/1b203ba7a15a4e1486d38598da3e541b/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:41.818924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] RBAC: Authorizing `identity:create_grant(project_id=b58a5ff1900746f9949ae1b474a5c965, user_id=1b203ba7a15a4e1486d38598da3e541b, role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:41.820346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed42887-d3bc-4baa-a3b7-6d345072c262 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:41.893691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 566/1123] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:37:41 2022] PUT /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965/users/1b203ba7a15a4e1486d38598da3e541b/roles/0bb62a6e624c40b4832f54f88761160b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:37:43.757652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b504-1b59-4a27-a4b4-771bd7428fa0 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:43.758183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b504-1b59-4a27-a4b4-771bd7428fa0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:43.758183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934b504-1b59-4a27-a4b4-771bd7428fa0 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:43.759433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 558/1124] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:43.765102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:43.765537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:43.765649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:43.783852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:43.785098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:43.839807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:43.856123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 567/1125] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:44.126084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b504-1b59-4a27-a4b4-771bd7428fa0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:44.224578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934b504-1b59-4a27-a4b4-771bd7428fa0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:44.257762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:44.258385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:44.258663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:44.258934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] PATH_INFO: `/v3/roles/0bb62a6e624c40b4832f54f88761160b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:44.260314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] RBAC: Authorizing `identity:get_role(role_id=0bb62a6e624c40b4832f54f88761160b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:44.261717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:44.263650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 559/1126] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:44 2022] GET /identity/v3/roles/0bb62a6e624c40b4832f54f88761160b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:44.269588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:44.363848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c127615-94c2-4640-b97a-a364121fc186 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:44.398074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:44.398629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:44.398901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:44.399165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] PATH_INFO: `/v3/users/1b203ba7a15a4e1486d38598da3e541b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:44.400271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] RBAC: Authorizing `identity:get_user(user_id=1b203ba7a15a4e1486d38598da3e541b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:44.401733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:44.403498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 568/1127] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:44 2022] GET /identity/v3/users/1b203ba7a15a4e1486d38598da3e541b => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:44.409091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:44.463665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:44.506550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec3c4a-4295-4730-8654-a8b6c7faf9b1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:44.540522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:44.541147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:44.541426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:44.541699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:44.607185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:44.608616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:44.671312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:44.714988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 560/1128] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:44.721056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:44.777121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:44.821816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651a400-277b-448c-a76c-cc475d97eb41 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:44.854213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:44.854826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:44.855097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:44.855362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] PATH_INFO: `/v3/projects/b58a5ff1900746f9949ae1b474a5c965` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:44.857022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] RBAC: Authorizing `identity:get_project(project_id=b58a5ff1900746f9949ae1b474a5c965)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:44.858430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0042a-5e8d-45cf-a412-188b9a7e24c4 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:44.860029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 569/1129] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:44 2022] GET /identity/v3/projects/b58a5ff1900746f9949ae1b474a5c965 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:44.866360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:44.962364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42378f5-32c0-4f2a-9502-2b6888f7f3cf None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:44.995621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:44.996218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:44.996495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:44.996799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:44.998029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] RBAC: Adding query filter params (role.id=0bb62a6e624c40b4832f54f88761160b, scope.project.id=b58a5ff1900746f9949ae1b474a5c965, user.id=1b203ba7a15a4e1486d38598da3e541b) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:44.998334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:44.999767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d43e26-0b41-4906-8b5c-8953ec330a88 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:45.005539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 561/1130] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:44 2022] GET /identity/v3/role_assignments?user.id=1b203ba7a15a4e1486d38598da3e541b&scope.project.id=b58a5ff1900746f9949ae1b474a5c965&role.id=0bb62a6e624c40b4832f54f88761160b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:46.904890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a12-68d0-4bb6-a6a1-0c7c13891f3e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:46.905450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a12-68d0-4bb6-a6a1-0c7c13891f3e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:46.905450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df375a12-68d0-4bb6-a6a1-0c7c13891f3e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:46.906768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 570/1131] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:46.911907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:46.912314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:46.912455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:46.930828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:46.932016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:46.945351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:46.997337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 562/1132] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:47.263330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df375a12-68d0-4bb6-a6a1-0c7c13891f3e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:47.366513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df375a12-68d0-4bb6-a6a1-0c7c13891f3e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:47.388577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:47.399552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:47.400188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:47.400467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:47.400766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:47.401877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:47.403940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17937e7-2768-48aa-bed5-9f39b4fb4c55 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:47.405532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 571/1133] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:47.411402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:47.509046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba81102-b5dd-4c01-b302-e905ceca117e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:47.542600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:47.543123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:47.543395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:47.543657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:47.544286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:47.545740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eee967-6a18-4510-9f40-17c0f62055be None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:47.620031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:47.625801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 563/1134] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:37:47 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:49.619188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceeb8-69ad-4f0b-bbb6-5de0cb2d7437 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:49.619749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceeb8-69ad-4f0b-bbb6-5de0cb2d7437 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:49.619749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cceeb8-69ad-4f0b-bbb6-5de0cb2d7437 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:49.621070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 572/1135] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:49.626849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:49.627223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:49.627395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:49.645877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:49.647075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:49.712722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 564/1136] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:49.989166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cceeb8-69ad-4f0b-bbb6-5de0cb2d7437 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:50.089374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cceeb8-69ad-4f0b-bbb6-5de0cb2d7437 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:50.123177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:50.123790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:50.124067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:50.124340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:50.125482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:50.126955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff03474-3421-4d47-8e02-5961fd5342e9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:50.128506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 573/1137] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:37:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:50.134213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:50.230652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1d278-4fa6-4048-bb62-648a1788b72a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:50.258862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:50.264524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:50.274741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:50.274741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:50.274741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:50.274741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:50.274741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d68c5d-b973-4c51-8e96-e0eb3c776f8e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:50.361694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 565/1138] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:37:50 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:37:52.293454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e456-5ce9-455b-a4b6-2100e360c863 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:52.294048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e456-5ce9-455b-a4b6-2100e360c863 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:52.294048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2e456-5ce9-455b-a4b6-2100e360c863 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:52.295321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 574/1139] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:52.300952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:52.301376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:52.301501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:52.319125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:52.320300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:52.386772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 566/1140] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:52.668873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e456-5ce9-455b-a4b6-2100e360c863 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:52.727436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:52.768247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2e456-5ce9-455b-a4b6-2100e360c863 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:52.801606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:52.802219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:52.802497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:52.802767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:52.813205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:52.817854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:52.819015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:52.820309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 575/1141] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:52.825132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:52.926836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2de59-32d5-4492-8427-17279e93b782 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:52.943510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:52.960428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:52.961060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:52.961337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:52.961650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:52.962316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:52.962618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:52.963981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:52.971723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 567/1142] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:52.977584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:53.074614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc48bab-099f-4ecd-bc24-053e474be47c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:53.090981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:53.107273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:53.107885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:53.108157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:53.108425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] PATH_INFO: `/v3/users/3791552ce2254434bfd918be890948ad` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:53.122181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] RBAC: Authorizing `identity:get_user(user_id=3791552ce2254434bfd918be890948ad)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:53.123594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:53.125179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 576/1143] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:52 2022] GET /identity/v3/users/3791552ce2254434bfd918be890948ad => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:53.131062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:53.231970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593da1-a4bd-43e6-b77c-8684ae6e4946 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:53.264129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:53.264743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:53.265022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:53.265292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:53.309612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:53.330178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:53.331614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:53.429543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 568/1144] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:53.435506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:53.535516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72913b61-f0ba-43ff-b0e2-0cc71fb2ed6e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:53.572515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:53.573167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:53.573443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:53.573713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] PATH_INFO: `/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:53.575478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] RBAC: Authorizing `identity:get_project(project_id=ca18ccc1cb904eaa86b3de981fd2ef13)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:53.576935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84be5d-2bf2-4bf4-8def-0ea10494ae6a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:53.578552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 577/1145] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:53 2022] GET /identity/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:53.584735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:53.660527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:53.681267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54bd833-1900-49e1-a421-730609ddb82f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:53.715751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:53.716345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:53.716622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:53.716925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:53.717644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=ca18ccc1cb904eaa86b3de981fd2ef13, user.id=3791552ce2254434bfd918be890948ad) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:53.717952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:53.719382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1880db-fad8-4ea2-895b-18ee28b5a0e6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:53.724621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 569/1146] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:53 2022] GET /identity/v3/role_assignments?user.id=3791552ce2254434bfd918be890948ad&scope.project.id=ca18ccc1cb904eaa86b3de981fd2ef13&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:55.609294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc132-f22a-46c9-8002-8964cda686b2 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:55.611300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc132-f22a-46c9-8002-8964cda686b2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:55.611300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc132-f22a-46c9-8002-8964cda686b2 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:55.611300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 578/1147] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:55.616310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:55.616783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:55.616879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:55.634572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:55.635882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:55.701283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 570/1148] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:55.968965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebc132-f22a-46c9-8002-8964cda686b2 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:56.014383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:56.068265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebc132-f22a-46c9-8002-8964cda686b2 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:56.101608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:56.102227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:56.102507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:56.102773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:56.112537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:56.117214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:56.118347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:37:56.119369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 579/1149] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:56.125055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:56.192636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:56.221143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a018a4-8e12-499b-a66f-fa2aa32f055f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:56.253975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:56.254560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:56.254834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:56.255098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:56.255753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:56.256048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:56.257415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:56.265762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 571/1150] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:56.271642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:56.367547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd1672-8c8c-4ab2-815f-08cd366e5c18 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:56.369807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:56.401839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:56.402478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:56.402757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:56.403032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] PATH_INFO: `/v3/users/3791552ce2254434bfd918be890948ad` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:56.404539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] RBAC: Authorizing `identity:get_user(user_id=3791552ce2254434bfd918be890948ad)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:56.405640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:56.407220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 580/1151] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:56 2022] GET /identity/v3/users/3791552ce2254434bfd918be890948ad => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:56.413011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:56.510233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39792d-88d0-4d16-8339-7d2190bf581f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:56.542980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:56.543596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:56.543910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:56.544180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] PATH_INFO: `/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:56.545294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] RBAC: Authorizing `identity:get_project(project_id=ca18ccc1cb904eaa86b3de981fd2ef13)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:56.546959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c03d9-f247-4809-b3b9-30964998aa9d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:56.548544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 572/1152] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:56 2022] GET /identity/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:56.554500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:56.651390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a04071b-c03d-4884-8e8c-a391ce6ca1ab None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:56.691626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:56.692284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:56.692563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:56.692867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] PATH_INFO: `/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13/users/3791552ce2254434bfd918be890948ad/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:56.694753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca18ccc1cb904eaa86b3de981fd2ef13, user_id=3791552ce2254434bfd918be890948ad, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:56.696195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2f334-6742-47cb-924c-3ba573549a72 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:56.796396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 581/1153] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:37:56 2022] PUT /identity/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13/users/3791552ce2254434bfd918be890948ad/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:37:58.712183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640215ef-712b-4672-ac08-79ca4701532d None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:58.712770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640215ef-712b-4672-ac08-79ca4701532d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:58.712770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640215ef-712b-4672-ac08-79ca4701532d None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:58.713978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 573/1154] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:37:58.719770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:58.720177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:58.720313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:58.738612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:37:58.739789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:37:58.765069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:58.804988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 582/1155] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:37:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:37:59.070921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640215ef-712b-4672-ac08-79ca4701532d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:59.073845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:59.171225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640215ef-712b-4672-ac08-79ca4701532d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:59.204737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:59.205345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:59.205618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:59.205881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:59.216043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:37:59.220796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:37:59.221895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:37:59.223005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 574/1156] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:37:59.227852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:59.325689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1289c6e-16b6-400f-b1de-57d274cea1ee None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:59.357832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:59.358455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:59.358728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:59.358995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:59.359651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:37:59.359961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:59.361352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:59.369080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 583/1157] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:59.375508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:59.436035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:59.473965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401d5d1-8045-46f1-9661-eb82df685901 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:59.506919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:59.507536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:59.507816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:59.508088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] PATH_INFO: `/v3/users/3791552ce2254434bfd918be890948ad` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:59.509217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] RBAC: Authorizing `identity:get_user(user_id=3791552ce2254434bfd918be890948ad)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:59.510704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:59.512281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 575/1158] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/users/3791552ce2254434bfd918be890948ad => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:59.518376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:59.543259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:59.617050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c616a4-aec7-4357-920f-6cea0c7fb9b7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:59.650034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:59.650633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:59.650909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:59.651179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:59.716338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:59.717812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:59.741063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:59.814501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 584/1159] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:37:59.820828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:37:59.918020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4e63e-49c0-4d23-b656-0e7f8225fa00 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:37:59.946216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:37:59.951720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:37:59.952331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:37:59.952605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:37:59.952904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] PATH_INFO: `/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:37:59.953983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] RBAC: Authorizing `identity:get_project(project_id=ca18ccc1cb904eaa86b3de981fd2ef13)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:37:59.955513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e4256-f993-492d-900a-8246a6341756 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:37:59.957139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 576/1160] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/projects/ca18ccc1cb904eaa86b3de981fd2ef13 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:37:59.963287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:00.061266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fac6c7-1666-4223-981d-50348adbab51 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:00.083160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:00.093863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:00.094463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:00.094741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:00.095013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:00.095740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=ca18ccc1cb904eaa86b3de981fd2ef13, user.id=3791552ce2254434bfd918be890948ad) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:00.096041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:00.097507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a43f8a-9b34-48eb-9927-2c94f405645c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:00.103120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 585/1161] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:37:59 2022] GET /identity/v3/role_assignments?user.id=3791552ce2254434bfd918be890948ad&scope.project.id=ca18ccc1cb904eaa86b3de981fd2ef13&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:02.022928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6a33-50c0-450c-8e56-28a713df4167 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:02.023483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6a33-50c0-450c-8e56-28a713df4167 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:02.023483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf6a33-50c0-450c-8e56-28a713df4167 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:02.024838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 577/1162] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:02.030742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:02.031135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:02.031299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:02.049361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:02.050542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:02.115719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 586/1163] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:02.381012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf6a33-50c0-450c-8e56-28a713df4167 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:02.481156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf6a33-50c0-450c-8e56-28a713df4167 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:02.513756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:02.514347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:02.514625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:02.514896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:02.578002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:02.581392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:02.582834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:02.678903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 578/1164] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:38:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:38:02.685522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:02.784758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e8497-8a97-4219-84c0-6f617ba4039a None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:02.801945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:02.818541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:02.819166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:02.819442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:02.819712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:02.826135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:02.827585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-f4b4cb64-6e96-4d39-81b0-8e9672c6eacd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:38:02.838459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:38:02.839675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:38:02.841098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 587/1165] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:38:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:38:02.846388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:02.939573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:02.943297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7515f-f2c6-483f-9edd-d0eb0c8c9ab7 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:02.976219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:02.976848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:02.977128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:02.977396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:02.978027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:02.978331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:02.979719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269da03-f4a1-49b1-959e-0ea6c0313dd9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:02.987454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 579/1166] 10.222.0.72 () {56 vars in 1115 bytes} [Wed Aug 24 07:38:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:04.858822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c39a-1c0f-4cda-a50a-1cc9528780cc None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:04.859364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c39a-1c0f-4cda-a50a-1cc9528780cc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:04.859364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01c39a-1c0f-4cda-a50a-1cc9528780cc None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:04.860808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 588/1167] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:04.866042 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f400ec-3e82-4c9a-8aeb-b8eb7b486298 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:04.866457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f400ec-3e82-4c9a-8aeb-b8eb7b486298 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:04.866587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f400ec-3e82-4c9a-8aeb-b8eb7b486298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:04.884784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-87f400ec-3e82-4c9a-8aeb-b8eb7b486298 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:04.885963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-87f400ec-3e82-4c9a-8aeb-b8eb7b486298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:04.950487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 580/1168] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:05.221961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01c39a-1c0f-4cda-a50a-1cc9528780cc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:05.321842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01c39a-1c0f-4cda-a50a-1cc9528780cc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:05.354331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:05.354918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:05.355194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:05.356010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:05.356702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:05.358179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9c6f1-3c47-4543-87d4-f5ad35087ccd None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:05.449289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 589/1169] 10.222.0.72 () {60 vars in 1138 bytes} [Wed Aug 24 07:38:05 2022] POST /identity/v3/domains => generated 267 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:38:07.408464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fcd8f-118e-4edf-b575-81c6a1b202f7 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:07.408891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fcd8f-118e-4edf-b575-81c6a1b202f7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:07.409041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fcd8f-118e-4edf-b575-81c6a1b202f7 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:07.410365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 581/1170] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:07.415632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:07.416061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:07.416166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:07.434875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:07.436064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:07.465309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:07.501705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 590/1171] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:07.773624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fcd8f-118e-4edf-b575-81c6a1b202f7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:07.864213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:07.874461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fcd8f-118e-4edf-b575-81c6a1b202f7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:07.907955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:07.908574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:07.908883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:07.909157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] PATH_INFO: `/v3/domains/4a25e1a5074d450bb37ec26b60029834` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:07.919770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a25e1a5074d450bb37ec26b60029834)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:07.921507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d08e4-2f60-4676-9c48-2a02ea10ca67 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:07.923076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 582/1172] 10.222.0.72 () {56 vars in 1183 bytes} [Wed Aug 24 07:38:07 2022] GET /identity/v3/domains/4a25e1a5074d450bb37ec26b60029834 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:07.928981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:08.025737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a00f2e-fdbb-40f5-9195-73ccc1e07c36 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:08.060025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:08.060620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:08.060923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:08.061191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:08.061938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:08.063366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1520ea8-fa01-4d8e-acc9-4a323df935fc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:08.149390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 591/1173] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:38:07 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:38:10.108001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7dbf8-a8ec-41b3-8ce8-9c28a19c23df None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:10.108637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7dbf8-a8ec-41b3-8ce8-9c28a19c23df None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:10.108637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7dbf8-a8ec-41b3-8ce8-9c28a19c23df None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:10.110006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 583/1174] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:10.115322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:10.115731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:10.115873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:10.133474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:10.134649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:10.149254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:10.202041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 592/1175] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:10.488915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7dbf8-a8ec-41b3-8ce8-9c28a19c23df None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:10.588675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7dbf8-a8ec-41b3-8ce8-9c28a19c23df None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:10.630558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:10.631200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:10.631479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:10.636078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] PATH_INFO: `/v3/domains/4a25e1a5074d450bb37ec26b60029834` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:10.636078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a25e1a5074d450bb37ec26b60029834)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:10.636078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc090bac-adb7-4be2-a304-8019db814a16 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:10.636078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 584/1176] 10.222.0.72 () {56 vars in 1183 bytes} [Wed Aug 24 07:38:10 2022] GET /identity/v3/domains/4a25e1a5074d450bb37ec26b60029834 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:10.642591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:10.744420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b680c2e-8158-47cc-b440-866f70c11ada None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:10.778601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:10.779210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:10.779488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:10.779758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:10.780399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:10.781905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b674f8a-07c9-4890-aff5-e8dd745e3853 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:10.885878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 593/1177] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:38:10 2022] POST /identity/v3/users => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:38:12.815304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54001aa7-e5bf-4481-ae6d-08c7532a3f4f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:12.815840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54001aa7-e5bf-4481-ae6d-08c7532a3f4f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:12.815840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54001aa7-e5bf-4481-ae6d-08c7532a3f4f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:12.817178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 585/1178] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:12.822411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:12.822804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:12.822956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:12.835754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:12.843198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:12.843198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:12.906822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 594/1179] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:13.173222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54001aa7-e5bf-4481-ae6d-08c7532a3f4f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:13.195392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:13.273346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54001aa7-e5bf-4481-ae6d-08c7532a3f4f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:13.305757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560792-1710-4549-b683-55b75350896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:13.306371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560792-1710-4549-b683-55b75350896d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.306648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560792-1710-4549-b683-55b75350896d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.306916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560792-1710-4549-b683-55b75350896d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.317076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560792-1710-4549-b683-55b75350896d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-ca560792-1710-4549-b683-55b75350896d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:38:13.321704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:38:13.322873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:38:13.324548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 586/1180] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:38:13.329485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:13.430469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928a283-cff6-4c73-9eba-08b99aa7b9a1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:13.463586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:13.464232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.464566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.464818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.465424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:13.465723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:13.467076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:13.469223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:13.475258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 595/1181] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:13.481130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560792-1710-4549-b683-55b75350896d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:13.560058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:13.579131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560792-1710-4549-b683-55b75350896d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:13.611667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:13.612271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.612571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.612867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] PATH_INFO: `/v3/users/f9f30aef01ca49b48b6a54ced87862ee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.626113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] RBAC: Authorizing `identity:get_user(user_id=f9f30aef01ca49b48b6a54ced87862ee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:13.627532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:13.629127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 587/1182] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/users/f9f30aef01ca49b48b6a54ced87862ee => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:13.635012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:13.735430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5956e1f-53ad-4b52-9966-66b6d2d21f44 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:13.767736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:13.768326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:13.768603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:13.768900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:13.830928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:13.834279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:13.835739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:13.930411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 596/1183] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:38:13.936979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:14.034439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fcad4-c336-434f-99a9-3bbc6cb2d1bc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:14.068049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:14.068689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:14.068963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:14.069238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] PATH_INFO: `/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:14.070343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d808ed4f4ed492195bbb4a0af989b4c)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:14.071765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65dae-b0e5-4a53-8686-757e6d2289d1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:14.073425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 588/1184] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:38:13 2022] GET /identity/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:14.081066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:14.173983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:14.177420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30665b-e256-40db-97e2-3cb55dbb61a5 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:14.210723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:14.211322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:14.211596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:14.211864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:14.212587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=7d808ed4f4ed492195bbb4a0af989b4c, user.id=f9f30aef01ca49b48b6a54ced87862ee) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:14.212922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:14.214870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb798663-37d8-465f-92e6-32c010884b38 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:14.220084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 597/1185] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:38:14 2022] GET /identity/v3/role_assignments?user.id=f9f30aef01ca49b48b6a54ced87862ee&scope.project.id=7d808ed4f4ed492195bbb4a0af989b4c&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:38:16.085723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdddd-e8a6-4840-a690-2c9a73ae3ed3 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:16.086272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdddd-e8a6-4840-a690-2c9a73ae3ed3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:16.086272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fdddd-e8a6-4840-a690-2c9a73ae3ed3 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:16.087595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 589/1186] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:16.092780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:16.093180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:16.093323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:16.111677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:16.112882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:16.176586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 598/1187] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:38:16.445424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fdddd-e8a6-4840-a690-2c9a73ae3ed3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:16.454953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:16.543903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fdddd-e8a6-4840-a690-2c9a73ae3ed3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:16.578523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:16.579130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:16.579403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:16.579667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:16.589369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:38:16.594048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:38:16.595138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:38:16.596176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 590/1188] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:38:16.601152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:16.689249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:16.697624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead4bda-58b1-4949-a826-6bee4d6bcfbb None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:16.731626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:16.732227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:16.732505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:16.732809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:16.733473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:16.733774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:16.735140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:16.742966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 599/1189] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:16.749401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:16.821339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:16.855468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8562-a773-4c66-865b-9f6e563cce95 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:16.888755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aec51-3065-4094-b13c-86328417aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:16.889374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aec51-3065-4094-b13c-86328417aabd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:16.889659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aec51-3065-4094-b13c-86328417aabd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:16.889926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aec51-3065-4094-b13c-86328417aabd None admin] PATH_INFO: `/v3/users/f9f30aef01ca49b48b6a54ced87862ee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:16.891027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aec51-3065-4094-b13c-86328417aabd None admin] RBAC: Authorizing `identity:get_user(user_id=f9f30aef01ca49b48b6a54ced87862ee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:16.892458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aec51-3065-4094-b13c-86328417aabd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:16.894056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 591/1190] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:38:16 2022] GET /identity/v3/users/f9f30aef01ca49b48b6a54ced87862ee => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:16.899820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:16.997893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65086bbb-4377-4c99-b331-a1d656112525 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:17.029853 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:17.030465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:17.030741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:17.031009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] PATH_INFO: `/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:17.032107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d808ed4f4ed492195bbb4a0af989b4c)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:17.033549 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517d4f48-a613-40ed-b833-db3e7062bdf9 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:17.035152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 600/1191] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:38:16 2022] GET /identity/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:17.041289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aec51-3065-4094-b13c-86328417aabd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:17.138219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aec51-3065-4094-b13c-86328417aabd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:17.171487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:17.172128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:17.172402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:17.172690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] PATH_INFO: `/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c/users/f9f30aef01ca49b48b6a54ced87862ee/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:17.174638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d808ed4f4ed492195bbb4a0af989b4c, user_id=f9f30aef01ca49b48b6a54ced87862ee, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:17.176051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe673c2d-2ecd-49b8-8992-ef56e12b1590 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:17.178225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:17.274085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 592/1192] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:38:17 2022] PUT /identity/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c/users/f9f30aef01ca49b48b6a54ced87862ee/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:38:19.135160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1099f1-0a84-4bdb-a2d0-c53b2b79475c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:19.136245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1099f1-0a84-4bdb-a2d0-c53b2b79475c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:19.136342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1099f1-0a84-4bdb-a2d0-c53b2b79475c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:19.137675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 601/1193] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:38:19.142948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc1b0-3962-4731-a049-14234a103394 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:19.143369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc1b0-3962-4731-a049-14234a103394 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:19.143499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bc1b0-3962-4731-a049-14234a103394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:19.161847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-610bc1b0-3962-4731-a049-14234a103394 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:38:19.163042 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-610bc1b0-3962-4731-a049-14234a103394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:38:19.227843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 593/1194] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:38:19.495777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1099f1-0a84-4bdb-a2d0-c53b2b79475c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:19.520506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:19.595162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1099f1-0a84-4bdb-a2d0-c53b2b79475c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:19.628099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:19.628748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:19.629034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:19.629304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:19.639763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:38:19.644571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:38:19.646164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:38:19.647267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 602/1195] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:38:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:38:19.652148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bc1b0-3962-4731-a049-14234a103394 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:19.750487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bc1b0-3962-4731-a049-14234a103394 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:19.773427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:19.783784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:19.784382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:19.784679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:19.784954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:19.785618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:19.785917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:19.787274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:19.795752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 594/1196] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:19.801662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:19.883632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:19.900915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826770b-726e-458f-9736-cd55bcb0988b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:19.935846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:19.936463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:19.936771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:19.937046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] PATH_INFO: `/v3/users/f9f30aef01ca49b48b6a54ced87862ee` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:19.938158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f30aef01ca49b48b6a54ced87862ee)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:19.939580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:19.941167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 603/1197] 10.222.0.72 () {56 vars in 1177 bytes} [Wed Aug 24 07:38:19 2022] GET /identity/v3/users/f9f30aef01ca49b48b6a54ced87862ee => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:19.947033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:20.043309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc13f5-5d6b-48c6-a946-0db40f77a4fb None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:20.075721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:20.076307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:20.076581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:20.076878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:20.128983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:20.141265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:20.142690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:20.239583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 595/1198] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:38:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:38:20.245628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:20.342616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20682c39-3a92-4945-a8be-ce8fef3b5b56 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:20.375265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:20.375865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:20.376139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:20.376407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] PATH_INFO: `/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:20.377551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] RBAC: Authorizing `identity:get_project(project_id=7d808ed4f4ed492195bbb4a0af989b4c)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:20.378978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b69d2-6fea-46f1-9211-e6a1398b71e1 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:20.380772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 604/1199] 10.222.0.72 () {56 vars in 1186 bytes} [Wed Aug 24 07:38:20 2022] GET /identity/v3/projects/7d808ed4f4ed492195bbb4a0af989b4c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:20.386750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:38:20.469306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:20.484139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f098f5-b6f1-4d0a-a998-328a08cca25c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:38:20.517629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:38:20.518222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:38:20.518502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:38:20.518774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:38:20.519502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=7d808ed4f4ed492195bbb4a0af989b4c, user.id=f9f30aef01ca49b48b6a54ced87862ee) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:38:20.519810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:38:20.521267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43818855-725a-4721-b18f-61d08db8bb1d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:38:20.527405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 596/1200] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:38:20 2022] GET /identity/v3/role_assignments?user.id=f9f30aef01ca49b48b6a54ced87862ee&scope.project.id=7d808ed4f4ed492195bbb4a0af989b4c&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:38:30.371068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:38:30.524344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=25 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:38.572699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aebf7-b5b4-40f0-8819-5d434ed3e15b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:38.573282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aebf7-b5b4-40f0-8819-5d434ed3e15b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:38.573282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aebf7-b5b4-40f0-8819-5d434ed3e15b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:38.574582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 605/1201] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:38.580036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:38.580446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:38.580584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:38.601487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:38.602690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:38.677258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 597/1202] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:38.950747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aebf7-b5b4-40f0-8819-5d434ed3e15b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:39.053208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aebf7-b5b4-40f0-8819-5d434ed3e15b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:39.086862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:39.087487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:39.087765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:39.088037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:39.098667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:39.100101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:39.114314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:39.115520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:39.116991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:39.116991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:39.116991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:39.116991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 606/1203] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:39.121461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:39.219343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c01aab-64db-41d8-931b-44fd8b04a5df None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:39.247957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:39.257801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:39.258423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:39.258701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:39.258965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:39.259955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:39.260498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:39.262421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f056a-a1d8-43df-a9bf-1460aebcac50 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:39.276572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 598/1204] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:39.283118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:39.335796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:39.383402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc90be-aa16-4631-88c4-856a3080efac None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:39.417556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:39.418166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:39.418490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:39.418784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:39.419425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:39.420891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380dcc2-42ba-4ec1-b1e2-203186c8b23e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:39.566906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 607/1205] 10.222.0.72 () {60 vars in 1132 bytes} [Wed Aug 24 07:39:39 2022] POST /identity/v3/users => generated 251 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:39:41.473754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffb5d-f9e3-4bff-8459-99bad3089581 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:41.474299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffb5d-f9e3-4bff-8459-99bad3089581 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:41.474419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ffb5d-f9e3-4bff-8459-99bad3089581 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:41.475667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 599/1206] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:41.480915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:41.481320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:41.481501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:41.503346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:41.504541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:41.571304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 608/1207] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:41.838687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ffb5d-f9e3-4bff-8459-99bad3089581 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:41.941139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ffb5d-f9e3-4bff-8459-99bad3089581 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:41.974456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:41.975081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:41.975401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:41.975674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:41.986913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:39:41.991670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:41.992886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:41.993842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 600/1208] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:41.998911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:42.038230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:42.097771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86a9e0-a410-46c6-9f26-a6f22813e545 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:42.129862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:42.130850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:42.130850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:42.131041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:42.131610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:42.131907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:42.133438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:42.141916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 609/1209] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:42.147455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:42.206531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:42.257798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128a6c-81e0-4025-ac05-a9d4ecb8c3f4 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:42.298066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:42.299714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:42.299714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:42.299714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:42.312388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:42.318913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:42.330994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:42.332152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:42.333077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:42.333077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:42.333077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:42.333779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 601/1210] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:42.339760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:42.452246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:42.458669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398897-522a-49cf-b08a-2eaec2a10c0a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:42.496238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:42.496945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:42.497130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:42.497906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:42.498546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:42.498843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:42.503605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:42.515421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 610/1211] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:42.521597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:42.640774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a89e-8fc6-4a2e-b51c-5eb0ba180d46 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:42.642488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:42.676687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:42.677255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:42.677525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:42.677788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:42.686205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:42.688168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:39:42.699334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:42.700428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:42.710637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 602/1212] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:42.711680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:42.835787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ef580-4aa9-4d81-84ef-d15801e7ec00 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:42.887179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:42.888400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:42.888708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:42.889040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:42.889846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:42.890148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:42.891647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:42.913608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 611/1213] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:39:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:42.920394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:43.034784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77746d2-0d9e-4f38-a7ab-46329e284fa6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:43.070734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:43.071328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:43.071601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:43.071874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:43.100741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:43.137450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:43.138946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:43.246600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 603/1214] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:39:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:39:43.252550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:43.271650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:43.352465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f396d-ddd3-48a5-b694-a158b90edde3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:43.387289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:43.387927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:43.388204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:43.388476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:43.398502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:43.399955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:43.415515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:43.423672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:43.424588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:43.424588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:43.424588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:43.424588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 612/1215] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:43.424881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:43.483374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:43.520510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255efc0-5df0-4d2c-954d-3534cde055a1 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:43.562792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:43.563487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:43.563666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:43.563938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:43.564590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:43.564928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:43.566296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:43.578996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 604/1216] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:43.584670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:43.655690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:43.685877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d65ad5-9ff3-4c39-bb0b-8e9043c5e048 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:43.719712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:43.720331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:43.720610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:43.720914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:43.727353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:43.728823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:39:43.739701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:43.745903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:43.747041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 613/1217] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:39:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:43.747673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:43.847050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd790038-2fa0-453c-a89c-abef040ff5b6 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:43.868809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:43.883933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:43.884552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:43.886600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:43.886957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:43.889663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:43.889663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:43.890192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9bc92-b2f6-4adf-b71f-80cc0fbf3713 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:43.904944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 605/1218] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:39:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:43.914916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:44.011840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62541a3-7a36-44eb-9f72-e505bcbb1958 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:44.040729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:44.046608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:44.047210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:44.047489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:44.047763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:44.048490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=14ac835d2376402f82fb9618a92f978e) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:44.048828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:44.050261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b9ed7-36d7-44e8-8931-76aaa2e751e2 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:44.055549 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 614/1219] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:39:43 2022] GET /identity/v3/role_assignments?user.id=14ac835d2376402f82fb9618a92f978e&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:45.954683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b558-806d-4fa0-820b-2117792b35db None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:45.955257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b558-806d-4fa0-820b-2117792b35db None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:45.955257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b558-806d-4fa0-820b-2117792b35db None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:45.956562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 606/1220] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:45.961757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:45.962235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:45.962333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:45.981901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:45.985056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:46.050451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 615/1221] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:46.323468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354b558-806d-4fa0-820b-2117792b35db None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:46.425548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354b558-806d-4fa0-820b-2117792b35db None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:46.459472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:46.460098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:46.460375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:46.460645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:46.471107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:39:46.485338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:46.486401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:46.487269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 607/1222] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:46.487990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:46.580225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddf29d-32bd-447c-b086-6f80c4267668 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:46.602929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:46.613782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:46.614386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:46.614665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:46.614937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:46.615597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] RBAC: Adding query filter params (name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:46.615900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:46.617295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:46.625740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 616/1223] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:46.631505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:46.675338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:46.739870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e83914-790d-40d6-bd02-3ea7c2f164bd None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:46.772574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:46.773289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:46.773561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:46.773828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:46.783520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:46.784971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:46.799826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:46.800992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:46.802192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:46.802192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:46.802192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:46.802192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 608/1224] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:46.806814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:46.905532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca7ad3-b8af-4f16-ba5d-dd48e8bec78d None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:46.938548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:46.939149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:46.939428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:46.939702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:46.940340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:46.940643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:46.942036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:46.953429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 617/1225] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:46.959179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.052526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:47.059015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7f9e6-a1e9-4e1c-acc3-6d9d60a56a5c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.091665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.092283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.092563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.092865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.100032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.101502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:39:47.112946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:47.119336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:47.120533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 609/1226] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:47.121302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.169946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:47.218961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e256ac-8d1b-49f4-9f6f-4c00cbbeb9cc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.252074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.252693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.252976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.253247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.253971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:47.254277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.255699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.273567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 618/1227] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:39:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:47.279926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.383771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87217bda-51cb-49de-ade9-d743996b1c88 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.418761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.419390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.419664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.419929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.430531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.431903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:47.445740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:47.446843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:47.447807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:47.447807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:47.447807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:47.447807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 610/1228] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:47.453422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.551937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aed4c9-59fa-4c6a-884b-5be955577277 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.554225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:47.585242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.585855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.586133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.586405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.587045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:47.587348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.588744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.600286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 619/1229] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:47.606223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.615447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:47.707319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956461c-96fd-49af-a7b7-cab660d083ee None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.746122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.746748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.747534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.747806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.754266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.755702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:39:47.769274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:47.770592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:47.771679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 611/1230] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:39:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:47.776297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.875412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754ef83-762e-4651-b53f-736d55660d4f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:47.908698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:47.909297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:47.909570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:47.909839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:47.910521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:47.910822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:47.912235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9b785-4740-43a5-9e5c-6e88417b1d39 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:47.924377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 620/1231] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:47.930444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:47.996824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:48.030378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0b099-0b0b-440c-ab03-c28294d1b143 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:48.063804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:48.064454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:48.064761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:48.065038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/14ac835d2376402f82fb9618a92f978e/roles/551908c203f04b568c5ff442ea75eb4e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:48.078736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=14ac835d2376402f82fb9618a92f978e, role_id=551908c203f04b568c5ff442ea75eb4e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:48.080174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20d292-f1f5-4a86-b44b-afa8f9950c9f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:48.176671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 612/1232] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:39:47 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/14ac835d2376402f82fb9618a92f978e/roles/551908c203f04b568c5ff442ea75eb4e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:39:50.062298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7fd-a8ed-4d01-8785-5c6d687e4ee4 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:50.062849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7fd-a8ed-4d01-8785-5c6d687e4ee4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:50.062849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3d7fd-a8ed-4d01-8785-5c6d687e4ee4 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:50.064182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 621/1233] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:50.069633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:50.070060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:50.070178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:50.088317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:50.089516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:50.156659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 613/1234] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:50.422348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3d7fd-a8ed-4d01-8785-5c6d687e4ee4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:50.448237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:50.521974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3d7fd-a8ed-4d01-8785-5c6d687e4ee4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:50.554583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:50.555192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:50.555463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:50.555727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:50.565854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:39:50.570575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:50.571668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:50.572862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 622/1235] 10.222.0.72 () {56 vars in 1102 bytes} [Wed Aug 24 07:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:50.577805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:50.623862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:50.675523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb41c24-36f6-43e9-ac3d-95106e2a915f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:50.711506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:50.712174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:50.712462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:50.713033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:50.713863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] RBAC: Adding query filter params (name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:50.714544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:50.716495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:50.724437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 614/1236] 10.222.0.72 () {56 vars in 1103 bytes} [Wed Aug 24 07:39:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:50.730223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:50.813864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:50.829341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ce02-5516-4eec-8ff1-daab5e3c14d2 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:50.864957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:50.865584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:50.865865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:50.866139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:50.875826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:50.877309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:50.891480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:50.897596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:50.900317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:50.900317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:50.900317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:50.900317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 623/1237] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:50.900608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.013689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ee457-085e-4f3e-aaf2-1ee35922ee15 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:51.022374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:51.046865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:51.047469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:51.047768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:51.048033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:51.048688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:51.048989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:51.050344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:51.062267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 615/1238] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:39:51.068245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.168917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93512a63-0c16-40c3-8ac1-89be4ca438e7 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:51.202250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:51.202867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:51.203144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:51.203415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:51.206026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:51.210824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:51.212276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:39:51.223765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:51.224906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:51.225962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 624/1239] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 07:39:51.230833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.339827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7861e6-2d14-4cb0-bcf8-9a58164568c0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:51.377770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:51.378373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:51.378651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:51.378927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:51.379649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:51.379954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:51.381407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:51.398892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 616/1240] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:39:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:51.404678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.504094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b9cdf-a06b-47c5-b943-08f0f6f1ce2f None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:51.537677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:51.538560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:51.538841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:51.539113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:51.599435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:51.605574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:51.607051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:51.712624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 625/1241] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:39:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:39:51.719077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.746510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:51.816336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9c1b-c352-4c2a-bc3d-7df4efba2708 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:51.851411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:51.851969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:51.852246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:51.852514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:51.862323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:51.863748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:51.877887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:51.883628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:51.885340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:51.885340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:51.885340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:51.885340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 617/1242] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:51.885776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:51.980289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:51.986721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d8348-8492-4fd9-a4dc-4705ab37fb0a None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:52.020117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:52.021001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:52.021281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:52.021554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:52.022189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:52.022490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:52.024705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:52.036609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 626/1243] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:52.042627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:52.119340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:52.141538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96756a3-efc4-44cc-b2a4-27a255a99430 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:52.173693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:52.174345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:52.174658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:52.174930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:52.181528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:52.182952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:39:52.195244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:52.196192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:52.197145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 618/1244] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:52.202951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:52.300800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b0e0-8894-459d-89d2-2155a611a522 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:52.333943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:52.334539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:52.334819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:52.335265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:52.335777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:52.336083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:52.337537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c3448-9607-4b3a-9dc7-8fdd0707a2ba None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:52.349143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 627/1245] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:39:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:52.355478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:52.455809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da43a5-16bf-4eed-9bdf-2421df5ffc4c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:52.488371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:52.489001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:52.489285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:52.489548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:52.490271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] RBAC: Adding query filter params (role.id=551908c203f04b568c5ff442ea75eb4e, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=14ac835d2376402f82fb9618a92f978e) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:52.490576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:52.492097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedd67-8cd1-4633-a26f-3e1abdd5aea8 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:52.498603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 619/1246] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:39:52 2022] GET /identity/v3/role_assignments?user.id=14ac835d2376402f82fb9618a92f978e&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=551908c203f04b568c5ff442ea75eb4e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:54.454553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c816a1-8db1-4aae-97dc-4fd3df860225 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:54.455118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c816a1-8db1-4aae-97dc-4fd3df860225 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:54.455118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c816a1-8db1-4aae-97dc-4fd3df860225 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:54.456623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 628/1247] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:54.462016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:54.462483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:54.462619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:54.469358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:54.481157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:54.482931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:54.547040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 620/1248] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:54.832672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c816a1-8db1-4aae-97dc-4fd3df860225 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:54.851547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:54.934097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c816a1-8db1-4aae-97dc-4fd3df860225 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:54.968323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:54.968986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:54.969261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:54.969529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:54.979713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:39:54.984381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:54.985404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:54.986298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 629/1249] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:54.991679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.092388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae042a-4cae-426d-b81e-57180b8904cc None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.120990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:55.126369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.126971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.127246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.127521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.128178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:55.128478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:55.129936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:55.137720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 621/1250] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:55.144193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.221391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:55.244111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813f3a9-b6ea-4a01-a1e9-323c2d5ec696 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.277628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.278249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.278524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.278794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.289503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:55.290942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:55.305277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:55.306395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:55.307484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:55.307484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:55.307484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:55.307484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 630/1251] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:55.312811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.428462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe7d4e-ab7b-4ab4-8f56-0dbe01ea4f79 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.461572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.462162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.462694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.462694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.463298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:55.463592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:55.465018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:55.476827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 622/1252] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:55.483103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.581167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc450c77-962f-44d5-b648-947b777333dc None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.606167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:55.614312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.614937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.615214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.615484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.623277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:55.624734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:39:55.635771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:55.644125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:55.645352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 631/1253] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:55.646019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.699781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:55.742702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784372-fa5d-412c-819a-6d9fae4f530c None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.777413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:55.785097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:55.799177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 623/1254] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:39:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:55.805012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:55.902469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebef332-d5a3-4407-a804-fabaa428e6b3 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:55.936445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:55.937058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:55.937337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:55.937608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:55.992275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:56.004174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.005648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.111209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 632/1255] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:39:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 07:39:56.117453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:56.214545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21d172-a56e-4dcf-94db-d32154430cb9 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:56.228094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:56.247802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:56.248357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:56.248628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:56.248921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:56.258335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.259755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:56.274198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:56.289290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:56.290294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:56.290294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:56.290294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:56.290294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 624/1256] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:56.290533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:56.366243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:56.377909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2bce1-490e-4a55-aaa9-e066f3536a69 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:56.415255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:56.415862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:56.416137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:56.416405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:56.417078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:56.417378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.418782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.430000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 633/1257] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:56.435972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:56.534750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e97430-b21c-4f69-be06-ebdb5f0c8abc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:56.567466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:56.568070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:56.568340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:56.568604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:56.576080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.581035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:39:56.588516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:56.589582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:39:56.590761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 625/1258] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:56.595616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:56.695224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cbedc-5af2-4be9-8533-7ce47a37fb86 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:56.728492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:56.729136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:56.729446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:56.729716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:56.730437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:56.730742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.732165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8727-92d9-4cfe-a9d0-772d99bf53a6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.744476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 634/1259] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:56.751098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:56.776942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:56.851585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320fadb-85ce-407c-9b8a-12844232bfb0 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:56.884447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:56.885070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:56.885351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:56.885623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:56.886350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=14ac835d2376402f82fb9618a92f978e) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:56.886659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:56.888762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736e043-eb73-4561-b7ca-443d87fd78dd None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:56.894102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 626/1260] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:39:56 2022] GET /identity/v3/role_assignments?user.id=14ac835d2376402f82fb9618a92f978e&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:58.799062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269acaa0-a4ec-4687-aefe-363b03d56329 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:58.799577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269acaa0-a4ec-4687-aefe-363b03d56329 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:58.799689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269acaa0-a4ec-4687-aefe-363b03d56329 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:58.801038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 635/1261] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:39:58.806158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:58.806548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:58.806700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:58.826904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:39:58.828081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:39:58.892900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 627/1262] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:39:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:39:59.173015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269acaa0-a4ec-4687-aefe-363b03d56329 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:59.183960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:59.274703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269acaa0-a4ec-4687-aefe-363b03d56329 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:59.327856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:59.328598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:59.328994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:59.329356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:59.340146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:39:59.345550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:59.346689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:59.347792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 636/1263] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:59.352676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:59.370584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:59.460833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3169bd3-ad34-45b5-b8df-f5038c11001c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:59.499495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:59.500103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:59.500593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:59.500915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:59.501593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:59.501902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:59.503274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:59.512003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 628/1264] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:39:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:59.518433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:59.592049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:59.627433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d78c9c-0c39-42fe-b655-033899d1cbff None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:59.672014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:59.672644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:59.672947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:59.673447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:59.685473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:59.688019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:39:59.706815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:39:59.711343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:39:59.712582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:39:59.712582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:39:59.712582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:39:59.712582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 637/1265] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:39:59.713795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:59.794493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:39:59.813352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a07703-032e-4b48-b51d-699f999f9962 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:59.846439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:39:59.847049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:39:59.847321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:39:59.847590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:39:59.848230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:39:59.848533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:39:59.849931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:39:59.861360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 629/1266] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:39:59.867090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:39:59.967649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a1056-4991-46ac-814e-44ec07f3272b None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:39:59.996129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.010618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.011240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.011511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.011777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.019206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.020643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:40:00.031459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:00.032578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:40:00.033774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 638/1267] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:39:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:00.038862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.139530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8383ea7-7a15-4523-a906-15831dfd2528 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.173473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.174078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.174362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.174667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.175428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:00.175733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.177204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.186510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.195713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 630/1268] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:00.201460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.303860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a09fd-590d-4b18-be29-fa0dccf00148 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.337770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.338395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.338672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.338941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.349808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.351225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:40:00.365510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:00.366650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:40:00.367629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:40:00.367629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:00.367629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:40:00.367629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 639/1269] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:00.372552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.470848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb966d-c99c-453c-a1cd-a73d3efe06fc None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.504003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.504635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.504941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.505210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.510325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:00.510325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.510325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.518356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 631/1270] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:00.524236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.556597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.622849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b8fe-db01-4cfc-bdbf-aafb57926524 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.656171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.656820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.657102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.657378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.664023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.665489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:40:00.677810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:00.683970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:40:00.685131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 640/1271] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:00.685739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.742527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.782913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a73d0-b437-489c-a6a9-9b2b6ea08a2f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.815297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.815915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.816096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.816367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.817080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:00.817384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.818804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23548fef-d13d-4c04-9f6b-adce6a9ab542 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:00.831013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 632/1272] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:40:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:00.837162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:00.928370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:00.936838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5f1e-3cd6-4081-b184-559366740aec None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:00.971538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:00.972184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:00.972461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:00.972757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] PATH_INFO: `/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/14ac835d2376402f82fb9618a92f978e/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:00.974634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0abff26982f466b88fe4ae574be3a29, user_id=14ac835d2376402f82fb9618a92f978e, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:00.976088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971bfe9e-5509-42ab-b518-ac682277d913 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:01.072737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 641/1273] 10.222.0.72 () {58 vars in 1439 bytes} [Wed Aug 24 07:40:00 2022] PUT /identity/v3/projects/c0abff26982f466b88fe4ae574be3a29/users/14ac835d2376402f82fb9618a92f978e/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:40:02.947697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0350-a7f4-4410-8e07-55c7d6da7126 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.948230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0350-a7f4-4410-8e07-55c7d6da7126 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.948230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d0350-a7f4-4410-8e07-55c7d6da7126 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.949593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 633/1274] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:02.955429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:02.955852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:02.955966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:02.974555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:02.975748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:03.043738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 642/1275] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:40:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:40:03.311017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0350-a7f4-4410-8e07-55c7d6da7126 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.420634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d0350-a7f4-4410-8e07-55c7d6da7126 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.440891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.457838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.458454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.458726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.458988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.468821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 07:40:03.473488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:03.474490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:03.475346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 634/1276] 10.222.0.72 () {56 vars in 1096 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:03.480729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.559930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:03.577466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36937c1-8959-40c0-b836-a14f33b20149 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.611317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.611979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.612261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.612535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.613312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:03.613616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.615074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.623099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 643/1277] 10.222.0.72 () {56 vars in 1099 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:03.629134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.732282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9c9b6-71ab-4367-a4c6-1867dc3e53de None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.765132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.765755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.766032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.766302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.776589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.778626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:40:03.792538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:03.793820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:40:03.794935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:40:03.794935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:03.794935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:03.794935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 635/1278] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:03.799579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.900211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcb2c9-e861-4604-8b5a-ad93d447ad64 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:03.934724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:03.940773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:03.950460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 644/1279] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:03.956254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:03.987137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.054590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28147b9e-b6d9-4408-a2fd-40b0526b8124 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.090227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.090933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.091209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.091474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.098971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.100480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 07:40:04.111312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:04.112384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:04.113564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 636/1280] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:40:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:04.118332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.122677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.217119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75481c59-d59c-4664-a361-98c74155f388 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.250084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9785b77-a980-4496-a192-672e98838201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.250689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9785b77-a980-4496-a192-672e98838201 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.250969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9785b77-a980-4496-a192-672e98838201 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.251243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9785b77-a980-4496-a192-672e98838201 None admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.251966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9785b77-a980-4496-a192-672e98838201 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:04.252274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9785b77-a980-4496-a192-672e98838201 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.253742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9785b77-a980-4496-a192-672e98838201 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.272338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 645/1281] 10.222.0.72 () {56 vars in 1143 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:04.278186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.366636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.376506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6f369-b4ef-4f8b-8411-0e2e2cc2c8fa None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.410791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.411397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.411673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.411942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.477745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.479129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.567494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.577523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 637/1282] 10.222.0.72 () {58 vars in 1282 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 07:40:04.583342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9785b77-a980-4496-a192-672e98838201 None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.644760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:04.682706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9785b77-a980-4496-a192-672e98838201 None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.716355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.717009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.717283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.717549 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.727837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.729265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 07:40:04.743953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:04.747440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 07:40:04.748579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 07:40:04.748579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 07:40:04.748579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:40:04.748579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 646/1283] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:04.751440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:04.848305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dbccc-ee68-4e86-b21f-1ab1e08f973e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:04.884834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:04.885435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:04.885713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:04.885982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:04.886616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:04.886915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:04.888336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:04.901365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 638/1284] 10.222.0.72 () {56 vars in 1109 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:04.907676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:05.007800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de6fe1-efd4-4fc7-b6a1-6815d697ac8c None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:05.020912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:05.040434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:05.041075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:05.041349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:05.041619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:05.048621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:05.050066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 07:40:05.060965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:05.062011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: ERROR keystone.server.flask.application Aug 24 07:40:05.062910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 647/1285] 10.222.0.72 () {56 vars in 1111 bytes} [Wed Aug 24 07:40:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:05.068036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:05.118944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:05.167306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277275b8-b694-4eb6-b012-88cc55556c4f None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:05.200759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:05.201370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:05.201639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:05.201905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:05.202589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:05.202892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:05.204316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dfecd-4a91-464e-aa9b-fa65881b284d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:05.217668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 639/1286] 10.222.0.72 () {56 vars in 1148 bytes} [Wed Aug 24 07:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:05.223905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:05.322428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e5b72-e783-4a2c-8130-f7ece96ac66e None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:05.355732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:05.356375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:05.356679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:05.356977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:05.357710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] RBAC: Adding query filter params (role.id=bbdc5cf8e5174908a2f4dd5f7c9df958, scope.project.id=c0abff26982f466b88fe4ae574be3a29, user.id=14ac835d2376402f82fb9618a92f978e) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:05.358020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:05.359456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc1a19-7844-4bb9-8852-b94b29953ae6 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:05.365337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 648/1287] 10.222.0.72 () {56 vars in 1374 bytes} [Wed Aug 24 07:40:05 2022] GET /identity/v3/role_assignments?user.id=14ac835d2376402f82fb9618a92f978e&scope.project.id=c0abff26982f466b88fe4ae574be3a29&role.id=bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:07.285856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec8db-9043-4a94-b27a-b5fe16544d81 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:07.286393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec8db-9043-4a94-b27a-b5fe16544d81 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:07.286393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec8db-9043-4a94-b27a-b5fe16544d81 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:07.287992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 640/1288] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:07.293273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:07.293678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:07.293823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:07.311987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:07.316184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:07.342456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:07.379890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 649/1289] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:40:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:40:07.646807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ec8db-9043-4a94-b27a-b5fe16544d81 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:07.748427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ec8db-9043-4a94-b27a-b5fe16544d81 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:07.759440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:07.782748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:07.783433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:07.783715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:07.783991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:07.784571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:07.786026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:07.795325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:07.796616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:07.797778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 641/1290] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:40:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:07.802618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:07.899463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b472-e974-453f-8887-f2a9f40310d1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:07.933177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:07.933826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:07.934108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:07.934377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:07.935008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] RBAC: Adding query filter params (name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:07.935310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:07.936690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d997138-88d8-49b4-91e4-7f2c393d718c None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:07.941492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 650/1291] 10.222.0.72 () {56 vars in 1116 bytes} [Wed Aug 24 07:40:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:07.947458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:08.049622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa08cde-149a-4e1a-bf82-646156377c42 None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:08.082194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:08.082797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:08.083074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:08.083345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] PATH_INFO: `/v3/services` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:08.083978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:08.084280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:08.085674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7c6d9-bd4a-4307-9d48-b4175c5a81a6 None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:08.090530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 642/1292] 10.222.0.72 () {56 vars in 1116 bytes} [Wed Aug 24 07:40:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:09.969946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc62c7-f8b1-4057-a153-7b4e7fbc6256 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:09.970474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc62c7-f8b1-4057-a153-7b4e7fbc6256 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:09.970565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc62c7-f8b1-4057-a153-7b4e7fbc6256 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:09.971894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 651/1293] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:09.977268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f12c3-0d1b-4147-8886-848062f19d25 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:09.977695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f12c3-0d1b-4147-8886-848062f19d25 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:09.977788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f12c3-0d1b-4147-8886-848062f19d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:09.995943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-486f12c3-0d1b-4147-8886-848062f19d25 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:09.997219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-486f12c3-0d1b-4147-8886-848062f19d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:10.026222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:10.068581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 643/1294] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:40:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 07:40:10.335463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc62c7-f8b1-4057-a153-7b4e7fbc6256 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:10.367126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:10.437411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc62c7-f8b1-4057-a153-7b4e7fbc6256 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:10.471635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:10.472231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:10.472507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:10.472809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:10.473444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:10.474817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafb79a-f615-4533-a239-978b6adf7f39 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:10.640757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 652/1295] 10.222.0.72 () {60 vars in 1141 bytes} [Wed Aug 24 07:40:10 2022] POST /identity/v3/services => generated 246 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:12.544837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eb059-4f63-4c29-8f31-f16dee4c6961 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.545474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eb059-4f63-4c29-8f31-f16dee4c6961 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.545474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34eb059-4f63-4c29-8f31-f16dee4c6961 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.546712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 644/1296] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:12.551847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:12.552240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:12.552401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:12.570659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:12.575205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:12.643265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 653/1297] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:40:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:40:12.913212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eb059-4f63-4c29-8f31-f16dee4c6961 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.014385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34eb059-4f63-4c29-8f31-f16dee4c6961 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.048431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.049075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.049404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.049680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] PATH_INFO: `/v3/services/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.050265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.051632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:13.060840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:13.062160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:13.063307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 645/1298] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:40:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:13.068237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.086910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:13.168728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480bfed-5f1e-4867-ba7d-58e95be49486 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.201404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.202085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.202378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.202631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.203258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:13.203560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.204952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.209990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 654/1299] 10.222.0.72 () {56 vars in 1116 bytes} [Wed Aug 24 07:40:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:13.218586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.249700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:13.316856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e013f-6303-45f8-8271-7cb2a52fa26d None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.350652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.351252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.351529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.351797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.352492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] RBAC: Adding query filter params (interface=public, service_id=a8574569a1ce4546b1c91a8c3be3eff0, region_id=RegionOne) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:13.352831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.354197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90366bc-94c4-4669-8bd9-48eedafe3c1d None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.359837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 646/1300] 10.222.0.72 () {56 vars in 1251 bytes} [Wed Aug 24 07:40:13 2022] GET /identity/v3/endpoints?service_id=a8574569a1ce4546b1c91a8c3be3eff0&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:13.365804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:13.453605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:13.472476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac790ad-06cf-4b97-98c2-3b856b6000ca None admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:13.505677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:13.506293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:13.506571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:13.506842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:13.507454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:13.508839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9b843-864a-4335-8c66-aa1a2cd25e8e None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:13.516007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 655/1301] 10.222.0.72 () {56 vars in 1087 bytes} [Wed Aug 24 07:40:13 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:40:15.447886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0be267-dd45-46fc-803e-2ec8c0c7b063 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.448587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0be267-dd45-46fc-803e-2ec8c0c7b063 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.449538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0be267-dd45-46fc-803e-2ec8c0c7b063 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.450881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 647/1302] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:40:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:40:15.456309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.456766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.456901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.474892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:40:15.476075 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:40:15.547911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 656/1303] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:40:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 07:40:15.819143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0be267-dd45-46fc-803e-2ec8c0c7b063 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:15.913519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:15.920795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0be267-dd45-46fc-803e-2ec8c0c7b063 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:15.954391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:15.955013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:15.955292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:15.955562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] PATH_INFO: `/v3/services/placement` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:15.956138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:15.957531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 07:40:15.966195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 07:40:15.967349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application with Lock( Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self._enter() Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application return self.creator() Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application created_value = creator( Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: ERROR keystone.server.flask.application Aug 24 07:40:15.968345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 648/1304] 10.222.0.72 () {56 vars in 1117 bytes} [Wed Aug 24 07:40:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 07:40:15.973679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.048091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:16.070821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f3d9c-0281-4aba-95d0-e3305969fbb5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.105941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.108505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.108505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.108505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] PATH_INFO: `/v3/services` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.108505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:40:16.108505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.109393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609436f-3298-4fb0-a089-8602518201c0 None admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.114279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 657/1305] 10.222.0.72 () {56 vars in 1116 bytes} [Wed Aug 24 07:40:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:40:16.120334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:40:16.217015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22f77a-3ee1-4649-97e8-f54bc0efa00e None admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:40:16.250015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:40:16.250613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:40:16.250889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:40:16.251159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:40:16.251813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:40:16.253210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710932d-d61b-4fb9-94f0-41cde84868ac None admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:40:16.351845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 649/1306] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:40:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:40:26.134704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=30 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:40:26.347227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:24.649076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb736521-e7dd-4d82-9544-11cc53ca9911 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:24.649649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb736521-e7dd-4d82-9544-11cc53ca9911 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:24.649649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb736521-e7dd-4d82-9544-11cc53ca9911 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:24.653003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 658/1307] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:24.656175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827bd4-28e5-42ef-a432-1ba383fae5ae None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:24.656612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827bd4-28e5-42ef-a432-1ba383fae5ae None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:24.656847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28827bd4-28e5-42ef-a432-1ba383fae5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:24.744752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-28827bd4-28e5-42ef-a432-1ba383fae5ae None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:24.745919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-28827bd4-28e5-42ef-a432-1ba383fae5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:24.871893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 650/1308] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:25.193304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.193907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.194611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.196291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 659/1309] 10.222.0.72 () {54 vars in 965 bytes} [Wed Aug 24 07:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:25.205858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99456c-c34a-4db8-bf65-62d038c5a5b6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.206634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99456c-c34a-4db8-bf65-62d038c5a5b6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.207103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99456c-c34a-4db8-bf65-62d038c5a5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.373949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8f99456c-c34a-4db8-bf65-62d038c5a5b6 None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:25.375587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8f99456c-c34a-4db8-bf65-62d038c5a5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.406540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:25.466565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 651/1310] 10.222.0.72 () {58 vars in 1062 bytes} [Wed Aug 24 07:47:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:25.476927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:25.479073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.621306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2550a-2548-4811-9020-aa1ef69273ec None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:25.729950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:25.731293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:25.731719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:25.732133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:25.733975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:25.791981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:25.854323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:25.856187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e10f16-b70c-4803-a17b-8f72227bb2af service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:25.878653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 660/1311] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:28.170759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:28.171338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:28.171338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:28.172639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 652/1312] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:28.177953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758820a-1ced-4bfb-a43d-efbc01c15c53 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:28.178395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758820a-1ced-4bfb-a43d-efbc01c15c53 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:28.178495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758820a-1ced-4bfb-a43d-efbc01c15c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:28.252799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f758820a-1ced-4bfb-a43d-efbc01c15c53 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:28.254081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f758820a-1ced-4bfb-a43d-efbc01c15c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:28.359216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 661/1313] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:28.646254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:28.647852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:28.761126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36d517-663d-4c4b-83a0-8dbe7770ecd1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:28.831334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:28.833674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:28.834271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:28.834543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:28.834808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:28.836466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:28.946457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:28.947935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368baeec-61ce-4106-b9f7-702c7c41b1a3 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:28.965686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 653/1314] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:31.345275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f6543-f274-4f60-9a2e-2353c6f2c607 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:31.345851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f6543-f274-4f60-9a2e-2353c6f2c607 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:31.345851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f6543-f274-4f60-9a2e-2353c6f2c607 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:31.347196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 662/1315] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:31.352465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b53da4-a601-462e-8267-91ab0c0e495e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:31.352930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b53da4-a601-462e-8267-91ab0c0e495e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:31.353045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b53da4-a601-462e-8267-91ab0c0e495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:31.417925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-20b53da4-a601-462e-8267-91ab0c0e495e None None] MFA Rules not processed for user `c6a2704c79154a9ab284fa13a4e9e446`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:31.419120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-20b53da4-a601-462e-8267-91ab0c0e495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:31.469652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:31.514317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 654/1316] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:31.819255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:31.819537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:31.819811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:31.821190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 663/1317] 10.222.0.72 () {54 vars in 965 bytes} [Wed Aug 24 07:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:31.827002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb7a4-5e00-4c20-87e5-94a33d8754cd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:31.827290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb7a4-5e00-4c20-87e5-94a33d8754cd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:31.827568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb7a4-5e00-4c20-87e5-94a33d8754cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:31.902088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-269eb7a4-5e00-4c20-87e5-94a33d8754cd None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:31.903286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-269eb7a4-5e00-4c20-87e5-94a33d8754cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:31.977811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 655/1318] 10.222.0.72 () {58 vars in 1062 bytes} [Wed Aug 24 07:47:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:31.986546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:31.988063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:31.997649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:32.091415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032c29a-4f15-4ecc-8453-cb32548824e5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:32.161946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:32.162663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:32.162935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:32.163201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:32.164812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:32.225107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:32.270196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:32.272175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018e4682-03f0-445e-bb1c-1029a46846a5 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:32.289492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 664/1319] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:35.302113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:35.302722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:35.302722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:35.304324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 656/1320] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:35.309770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca667-fc36-4b92-9165-8bdab3049766 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:35.310155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca667-fc36-4b92-9165-8bdab3049766 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:35.310361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ca667-fc36-4b92-9165-8bdab3049766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:35.375607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3a2ca667-fc36-4b92-9165-8bdab3049766 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:35.376862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a2ca667-fc36-4b92-9165-8bdab3049766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:35.481123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 665/1321] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:35.766733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:35.768299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:35.819629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:35.872970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5fbaf-95b8-41ac-9116-a97c9e593963 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:35.947550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:35.948156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:35.948441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:35.948747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:35.950343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:36.043645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:36.057602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:36.059126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7748d-4998-4449-af44-30c75686c667 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:36.076777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 657/1322] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:38.984768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:38.985323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:38.985754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:38.988429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 666/1323] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:38.997178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7186-b55b-45d6-820c-df6f642113f9 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:38.997633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7186-b55b-45d6-820c-df6f642113f9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:38.997748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b7186-b55b-45d6-820c-df6f642113f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.063917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6d7b7186-b55b-45d6-820c-df6f642113f9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:39.065163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6d7b7186-b55b-45d6-820c-df6f642113f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.162511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 658/1324] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:39.446084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:39.447724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.468843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:39.558659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cba09-b854-40f5-afaf-929cc21dcb3b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:39.633753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:39.634360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:39.634635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:39.634902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:39.636496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:39.708683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:39.748289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:39.749672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca5b69-b805-4196-bf3c-5a0efa86d14d service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:39.768607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 667/1325] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:41.732197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:41.732752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:41.732752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:41.734057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 659/1326] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:41.739577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0aeec-9a73-4419-b4c9-c554a7d443fc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:41.740041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0aeec-9a73-4419-b4c9-c554a7d443fc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:41.740238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0aeec-9a73-4419-b4c9-c554a7d443fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:41.828391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d0e0aeec-9a73-4419-b4c9-c554a7d443fc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:41.829560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d0e0aeec-9a73-4419-b4c9-c554a7d443fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:41.928491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 668/1327] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:42.220698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:42.222233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:42.287097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:42.331970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecddab6-2317-4a98-903e-dbd8fee14c0f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:42.409478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:42.410109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:42.410382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:42.410648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:42.412344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:42.531702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:42.533913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:42.535344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0cad6-9906-4bfd-94f3-f71b49a868b6 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:42.555063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 660/1328] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:44.539930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.540466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.540466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.541867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 669/1329] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:44.547419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b612-db12-4a4e-bd8c-92b8afb1d102 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:44.548199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b612-db12-4a4e-bd8c-92b8afb1d102 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:44.548505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b612-db12-4a4e-bd8c-92b8afb1d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:44.615388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d183b612-db12-4a4e-bd8c-92b8afb1d102 None None] MFA Rules not processed for user `c6a2704c79154a9ab284fa13a4e9e446`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:44.616518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d183b612-db12-4a4e-bd8c-92b8afb1d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:44.718068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 661/1330] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:45.002374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:45.003955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.039748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:45.136568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398a7eb-d7a0-4390-b79b-11e4b1419d11 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:45.215658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:45.216266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:45.216539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:45.216838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:45.218430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:45.296258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:45.327246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:45.328757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe21bf5-8288-4631-ac3c-1de55264fe10 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:45.348230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 670/1331] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:47.685506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.686021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.686021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.687321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 662/1332] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:47.692697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308065f7-e1ba-4dbf-8d82-c5af8e0dfaeb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:47.693128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308065f7-e1ba-4dbf-8d82-c5af8e0dfaeb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:47.693310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308065f7-e1ba-4dbf-8d82-c5af8e0dfaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:47.758304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-308065f7-e1ba-4dbf-8d82-c5af8e0dfaeb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:47.759499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-308065f7-e1ba-4dbf-8d82-c5af8e0dfaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:47.855631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 671/1333] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:48.148942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:48.150489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:48.221017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:48.258894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05e34c-94d7-4094-b168-984b3f45f859 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:48.330966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:48.331566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:48.331846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:48.332118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:48.333765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:48.440377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:48.441845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee4326-8c5e-4d26-b5aa-b70c2c33cee8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:48.452263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:48.461715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 663/1334] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:50.429678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:50.430167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:50.430275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:50.431541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 672/1335] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:47:50.436731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f22ac-fd36-4a81-975d-475318032060 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:50.437175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f22ac-fd36-4a81-975d-475318032060 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:50.437284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f22ac-fd36-4a81-975d-475318032060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:50.502245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0e4f22ac-fd36-4a81-975d-475318032060 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:50.503464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0e4f22ac-fd36-4a81-975d-475318032060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:50.609833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 664/1336] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:50.905381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:50.906933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:50.946665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:51.010817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd652f-9078-4c3a-a992-b53e8a7e0e6e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:51.083219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:51.084369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:51.084670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:51.084950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:51.089212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:51.176957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:51.204514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:51.205978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8502c7-1876-4600-9b1c-a3c1ffecd083 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:51.227001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 673/1337] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:53.822059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.822633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.822736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.824087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 665/1338] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:53.829391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67250f5-6bdd-4995-b5b3-e2ad8b864b2d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:53.829863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67250f5-6bdd-4995-b5b3-e2ad8b864b2d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:53.829967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67250f5-6bdd-4995-b5b3-e2ad8b864b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:53.894303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b67250f5-6bdd-4995-b5b3-e2ad8b864b2d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:53.895491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b67250f5-6bdd-4995-b5b3-e2ad8b864b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:53.989026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 674/1339] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:54.268776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:54.270358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:54.345198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:54.372373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddd45b-11c2-4878-b066-3fbf4243f506 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:54.445744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:54.446338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:54.446611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:54.446876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:54.448447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:54.554226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:54.555652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe71b02-1fff-416f-b36a-a5dd0e431aa8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:54.574082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 666/1340] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:47:58.784023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:58.784592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:58.784592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:58.785931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 675/1341] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:47:58.791381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54fa8b-9d3c-4f76-b7bd-1b6e92df0607 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:58.791849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54fa8b-9d3c-4f76-b7bd-1b6e92df0607 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:58.791956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54fa8b-9d3c-4f76-b7bd-1b6e92df0607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:58.797501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:58.860333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dc54fa8b-9d3c-4f76-b7bd-1b6e92df0607 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:47:58.861532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dc54fa8b-9d3c-4f76-b7bd-1b6e92df0607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:58.961119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 667/1342] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:47:59.251226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:47:59.253037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:59.301377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:59.357374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd5d8c-823f-442b-bd1c-1a1cfe278dea None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:47:59.433403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:47:59.434006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:47:59.434279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:47:59.434545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:47:59.436184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:47:59.531194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:47:59.548005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:47:59.549500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d90a06-ab35-47ab-b20a-c953d329a217 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:47:59.568583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 676/1343] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:02.751406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:02.751940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:02.751940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:02.753291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 668/1344] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:02.758499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d529-5f95-4c7d-a726-91889d1cc670 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:02.758896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d529-5f95-4c7d-a726-91889d1cc670 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:02.759201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d529-5f95-4c7d-a726-91889d1cc670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:02.826420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e418d529-5f95-4c7d-a726-91889d1cc670 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:02.827638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e418d529-5f95-4c7d-a726-91889d1cc670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:02.924111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 677/1345] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:03.228482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:03.230053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:03.332601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:03.349944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a079564-93b6-4772-b985-07011171e6b4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:03.423678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:03.424282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:03.424558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:03.424854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:03.426441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:03.535359 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:03.536847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5625d4-853b-4df7-a956-1feaf1b16cf2 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:03.555684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 669/1346] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:07.985362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:07.985901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:07.985901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:07.991402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 678/1347] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:07.993668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e145d7-8fce-4e04-be8e-77e25c9b8266 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:07.994154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e145d7-8fce-4e04-be8e-77e25c9b8266 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:07.994261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e145d7-8fce-4e04-be8e-77e25c9b8266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.010652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:08.088044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-12e145d7-8fce-4e04-be8e-77e25c9b8266 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:08.089327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-12e145d7-8fce-4e04-be8e-77e25c9b8266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.201162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 670/1348] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:08.482925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:08.484729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.549125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:08.599461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d601-7327-45d5-aed0-483f21dfb896 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:08.674266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:08.674855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:08.675183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:08.675385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:08.677671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:08.789147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:08.792320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:08.793798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5989f5-44d8-4b32-ad82-b13a30c44b26 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:08.812900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 679/1349] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:13.804010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd523e42-080c-4d6f-8925-1da65064828e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.804456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd523e42-080c-4d6f-8925-1da65064828e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.804628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd523e42-080c-4d6f-8925-1da65064828e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.806602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 671/1350] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:13.813297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f16c-83f0-4201-82ff-cc17f4615694 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:13.813766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f16c-83f0-4201-82ff-cc17f4615694 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:13.813866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4f16c-83f0-4201-82ff-cc17f4615694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:13.921877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4dd4f16c-83f0-4201-82ff-cc17f4615694 None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:13.921877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4dd4f16c-83f0-4201-82ff-cc17f4615694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:14.088864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 680/1351] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:48:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:18.199067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=59 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:18.281896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b408-80d8-4d12-8f2d-3f6bb2764b49 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:18.282205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b408-80d8-4d12-8f2d-3f6bb2764b49 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:18.282484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271b408-80d8-4d12-8f2d-3f6bb2764b49 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:18.283952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 672/1352] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:18.289600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d156b0-07a0-43e7-a4de-78bd2bb4c7af None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:18.290072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d156b0-07a0-43e7-a4de-78bd2bb4c7af None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:18.290157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d156b0-07a0-43e7-a4de-78bd2bb4c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:18.346198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:18.369990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-58d156b0-07a0-43e7-a4de-78bd2bb4c7af None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:18.371202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-58d156b0-07a0-43e7-a4de-78bd2bb4c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:18.496467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 681/1353] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:48:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:19.755238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453dad-1c39-4d47-b67e-82a4b8a5e58e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:19.755767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453dad-1c39-4d47-b67e-82a4b8a5e58e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:19.755767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453dad-1c39-4d47-b67e-82a4b8a5e58e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:19.757034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 673/1354] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:19.762377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e577-5531-4d1b-8d4d-b7397ff4ad1a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:19.762811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e577-5531-4d1b-8d4d-b7397ff4ad1a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:19.762929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5e577-5531-4d1b-8d4d-b7397ff4ad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:19.840749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0fd5e577-5531-4d1b-8d4d-b7397ff4ad1a None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:19.842093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0fd5e577-5531-4d1b-8d4d-b7397ff4ad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:19.884122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:19.950480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 682/1355] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:48:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:23.291815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa778e-8c37-4d26-a2bd-c5b62aebf2f4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.292374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa778e-8c37-4d26-a2bd-c5b62aebf2f4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.292374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa778e-8c37-4d26-a2bd-c5b62aebf2f4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.293730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 674/1356] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:23.299062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b27-4682-40f2-80d0-5a02b434a126 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.299463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b27-4682-40f2-80d0-5a02b434a126 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.299604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b27-4682-40f2-80d0-5a02b434a126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.385399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5e764b27-4682-40f2-80d0-5a02b434a126 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:23.386598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5e764b27-4682-40f2-80d0-5a02b434a126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:23.527967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 683/1357] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:23.833076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.833366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.833801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.835166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 675/1358] 10.222.0.72 () {54 vars in 962 bytes} [Wed Aug 24 07:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:23.844062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3f32b-a048-4ed7-8d6c-39b6067d679c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:23.844350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3f32b-a048-4ed7-8d6c-39b6067d679c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:23.844681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead3f32b-a048-4ed7-8d6c-39b6067d679c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:23.860952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:23.923611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ead3f32b-a048-4ed7-8d6c-39b6067d679c None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:23.924864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ead3f32b-a048-4ed7-8d6c-39b6067d679c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.042518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 684/1359] 10.222.0.72 () {58 vars in 1059 bytes} [Wed Aug 24 07:48:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:24.051005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:24.052599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.229956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632f461-7331-43fe-a2f7-f383e1f7c6c0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:24.322190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:24.366743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:24.367514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:24.367897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:24.368292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:24.370144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:24.464200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:24.465887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac75ed-28d9-48e2-9ce1-8065fa748263 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:24.485994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 676/1360] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:48:26.626097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:26.626097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:26.626725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:26.629057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 685/1361] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:26.633870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cf4e-970a-4e12-8bfb-3163972a25ca None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:26.633870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cf4e-970a-4e12-8bfb-3163972a25ca None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:26.634667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cf4e-970a-4e12-8bfb-3163972a25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:26.706801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9080cf4e-970a-4e12-8bfb-3163972a25ca None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:26.708384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9080cf4e-970a-4e12-8bfb-3163972a25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:26.749564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:26.811724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 677/1362] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:27.108826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:27.111063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:27.216300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:27.309385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbc99-7554-4e16-9bdb-2e49793e6f4e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:27.438294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:27.438912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:27.439263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:27.439604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:27.441309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:27.465997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:27.542925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:27.544529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e01a02-2e28-4b6b-973c-c780aabfe141 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:27.569156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 686/1363] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:30.054566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcec6d-0357-4026-aea8-fcabf129d8bf None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:30.055141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcec6d-0357-4026-aea8-fcabf129d8bf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:30.055141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcec6d-0357-4026-aea8-fcabf129d8bf None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:30.056436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 678/1364] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:48:30.061856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b0eb-feea-421b-afc8-f766c0abc531 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:30.062280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b0eb-feea-421b-afc8-f766c0abc531 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:30.062376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919b0eb-feea-421b-afc8-f766c0abc531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:30.132725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9919b0eb-feea-421b-afc8-f766c0abc531 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:30.133933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9919b0eb-feea-421b-afc8-f766c0abc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:30.240020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:30.244695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 687/1365] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:30.532031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:30.532311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:30.532876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:30.534597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 679/1366] 10.222.0.72 () {54 vars in 962 bytes} [Wed Aug 24 07:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:30.540637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59ca2-6a07-43fd-9aa1-b6c3c7221ea6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:30.540973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59ca2-6a07-43fd-9aa1-b6c3c7221ea6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:30.541247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a59ca2-6a07-43fd-9aa1-b6c3c7221ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:30.622518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-58a59ca2-6a07-43fd-9aa1-b6c3c7221ea6 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:30.623723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-58a59ca2-6a07-43fd-9aa1-b6c3c7221ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:30.755766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 688/1367] 10.222.0.72 () {58 vars in 1059 bytes} [Wed Aug 24 07:48:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:30.763679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:30.765511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:30.923113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aad4e-2d04-4fe4-8081-dde18d95efd2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:30.945249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:31.048513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:31.049152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:31.049431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:31.049700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:31.051470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:31.143128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:31.145292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba11dee-16ae-4883-ac47-68767a335984 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:31.164321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 680/1368] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:48:31.644590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119b46f-bb0d-499d-9ecf-316489f5fd5c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:31.645686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119b46f-bb0d-499d-9ecf-316489f5fd5c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:31.645686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119b46f-bb0d-499d-9ecf-316489f5fd5c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:31.647210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 689/1369] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:31.652324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046fa0c-f61d-4fc8-8446-665786b528cd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:31.652612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046fa0c-f61d-4fc8-8446-665786b528cd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:31.652926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046fa0c-f61d-4fc8-8446-665786b528cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:31.686913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:31.725541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1046fa0c-f61d-4fc8-8446-665786b528cd None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:31.726839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1046fa0c-f61d-4fc8-8446-665786b528cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:31.830605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 681/1370] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:32.417763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:32.417763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:32.417763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:32.418832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 690/1371] 10.222.0.72 () {54 vars in 964 bytes} [Wed Aug 24 07:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:32.423712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b29be7-41e4-45c0-92a8-9f0bd1d2d066 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:32.424191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b29be7-41e4-45c0-92a8-9f0bd1d2d066 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:32.424360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b29be7-41e4-45c0-92a8-9f0bd1d2d066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:32.507212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e1b29be7-41e4-45c0-92a8-9f0bd1d2d066 None None] MFA Rules not processed for user `8a2fd7d0bbcb4a04861d7cf060c671a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:32.508454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e1b29be7-41e4-45c0-92a8-9f0bd1d2d066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:32.557189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:32.591503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 682/1372] 10.222.0.72 () {58 vars in 1061 bytes} [Wed Aug 24 07:48:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:32.601430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:32.602963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:32.658016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:32.724071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54798154-8100-42ae-8f64-7b12cad6f8d1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:32.802665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:32.803529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:32.803812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:32.804082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:32.805727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:32.921001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:32.938576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:32.940021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013c12e-033d-4152-994a-1b6c46c4e9b8 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:32.962928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 691/1373] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:33.331426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:33.331714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:33.331990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:33.333904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 683/1374] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:33.339434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b5b1-4232-4461-891a-fb60543b769a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:33.339725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b5b1-4232-4461-891a-fb60543b769a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:33.340002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b5b1-4232-4461-891a-fb60543b769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:33.412090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-93d7b5b1-4232-4461-891a-fb60543b769a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:33.413357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-93d7b5b1-4232-4461-891a-fb60543b769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:33.515466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 692/1375] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:33.805930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:33.807543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:33.959916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496988a5-7035-4573-b8fc-fb4644fdfdf1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:34.010720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:34.074582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:34.075180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:34.075454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:34.075719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:34.077336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:34.165808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:34.167396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253bc3f-997d-49ea-908f-702211d1d4b5 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:34.186830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 684/1376] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:36.275359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:36.275885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:36.275885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:36.277368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 693/1377] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:36.282810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33417e-e75f-4ac2-ae20-7cee5aabd6cc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:36.283248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33417e-e75f-4ac2-ae20-7cee5aabd6cc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:36.283355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33417e-e75f-4ac2-ae20-7cee5aabd6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:36.368612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7e33417e-e75f-4ac2-ae20-7cee5aabd6cc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:36.369806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7e33417e-e75f-4ac2-ae20-7cee5aabd6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:36.376753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:36.471469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 685/1378] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:36.768335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:36.772126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:36.849972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:36.957170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79fdf3-8abd-4ef9-bc6f-6df71ba45d66 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:37.096150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:37.096930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:37.097206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:37.097477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:37.099088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:37.119166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:37.193903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:37.195503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a3b9b-8767-4111-b7d9-a017a541e3f9 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:37.214580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 694/1379] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:39.277440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:39.277992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:39.277992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:39.279371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 686/1380] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:39.284907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd33b-3283-48d7-bc6e-dc5632e127fb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:39.285939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd33b-3283-48d7-bc6e-dc5632e127fb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:39.286017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd33b-3283-48d7-bc6e-dc5632e127fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:39.355190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-477fd33b-3283-48d7-bc6e-dc5632e127fb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:39.356290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-477fd33b-3283-48d7-bc6e-dc5632e127fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:39.462208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:39.476756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 695/1381] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:39.765524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:39.767169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:39.919372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:39.926860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe5db9-570d-4834-bfac-b0a7fac1aaec None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:40.052155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:40.052787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:40.053330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:40.053603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:40.055199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:40.154254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:40.156897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0261a9a8-1a77-43f1-9d59-072b97d73529 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:40.168511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:40.178683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 687/1382] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:42.272581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:42.273183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:42.273290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:42.274730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 696/1383] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:42.280395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37352a7c-600e-468c-8c48-854162ad5efc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:42.280844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37352a7c-600e-468c-8c48-854162ad5efc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:42.281125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37352a7c-600e-468c-8c48-854162ad5efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:42.371361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-37352a7c-600e-468c-8c48-854162ad5efc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:42.374865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-37352a7c-600e-468c-8c48-854162ad5efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:42.484961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 688/1384] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:42.782495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:42.784063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:42.941231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26d102-8b33-4995-92aa-9e8a0611260c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:43.009927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:43.059866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:43.060677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:43.061048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:43.061415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:43.063474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:43.171398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:43.173801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0dcee-54d6-4049-9d3f-3344efd0a6fe service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:43.197163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 697/1385] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:45.200250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:45.200844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:45.200945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:45.202292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 689/1386] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:45.208274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736ae13-4989-4cf1-9e5f-86d5020695cb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:45.208872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736ae13-4989-4cf1-9e5f-86d5020695cb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:45.208872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736ae13-4989-4cf1-9e5f-86d5020695cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:45.278888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0736ae13-4989-4cf1-9e5f-86d5020695cb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:45.280098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0736ae13-4989-4cf1-9e5f-86d5020695cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:45.314340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:45.379610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 698/1387] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:45.674682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:45.676326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:45.755819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:45.840645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4bec4-390f-4ae9-a0eb-605863fdb0c4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:45.966856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:45.967403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:45.967709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:45.968005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:45.969696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:46.010768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:46.065449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:46.067710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88f114-f66e-4a5d-9397-96c5de8fd65d service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:46.087073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 690/1388] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:48.215413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672d57-5e4d-4e5d-b92d-3c15c996bc35 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.216065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672d57-5e4d-4e5d-b92d-3c15c996bc35 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.216065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1672d57-5e4d-4e5d-b92d-3c15c996bc35 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.217338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 699/1389] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:48.223078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aca-2cf5-4923-b216-a74ed07f8c3d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.223524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aca-2cf5-4923-b216-a74ed07f8c3d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.223614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aca-2cf5-4923-b216-a74ed07f8c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.291839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-19622aca-2cf5-4923-b216-a74ed07f8c3d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:48.293108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-19622aca-2cf5-4923-b216-a74ed07f8c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:48.391390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 691/1390] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:48.425411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.425781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.426144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.427582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 700/1391] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:48.433096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb33e-7a81-4c31-a312-5a9944df1842 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.433393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb33e-7a81-4c31-a312-5a9944df1842 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.433666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb33e-7a81-4c31-a312-5a9944df1842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.449527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:48.501028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-081fb33e-7a81-4c31-a312-5a9944df1842 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:48.502242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-081fb33e-7a81-4c31-a312-5a9944df1842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:48.598701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 692/1392] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:48.681253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:48.682823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:48.834328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7fd7-e996-4800-8746-88ee512795e8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:48.850704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:48.902490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac52c9-e04e-48ce-be37-fd048663b960 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.902770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac52c9-e04e-48ce-be37-fd048663b960 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.903036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac52c9-e04e-48ce-be37-fd048663b960 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.904450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 693/1393] 10.222.0.72 () {54 vars in 967 bytes} [Wed Aug 24 07:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:48:48.909392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.909707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.909968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.954582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:48.955192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:48.955466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:48.955729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:48.957433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:48.986173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] MFA Rules not processed for user `b95e296a3e2f48599b916a0ac7be2213`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:48.987366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:49.022131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:49.047805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:49.049844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f665d8-4f10-41fb-b35f-c5173d6cf8aa service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:49.066895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 694/1394] 10.222.0.72 () {58 vars in 1064 bytes} [Wed Aug 24 07:48:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:48:49.071072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 701/1395] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:49.074772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:49.082554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3eb2fb-a44b-48a9-a0f4-db7b9c995758 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:49.158206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:49.158813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:49.159091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:49.159360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:49.161062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:49.273520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:49.277565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:49.279265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cced4-d64b-48f0-86c6-0ce57f3edd0e service glance] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:49.300156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 695/1396] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:48:50.176403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda7e31-1521-4906-9ee6-4b5e2e1ae935 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:50.180576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda7e31-1521-4906-9ee6-4b5e2e1ae935 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:50.180576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda7e31-1521-4906-9ee6-4b5e2e1ae935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:50.202401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:50.244758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-abda7e31-1521-4906-9ee6-4b5e2e1ae935 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:50.246582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-abda7e31-1521-4906-9ee6-4b5e2e1ae935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:50.336929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 702/1397] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:48:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:50.384066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:50.384549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:50.384644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:50.385992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 696/1398] 10.222.0.72 () {54 vars in 943 bytes} [Wed Aug 24 07:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:50.392434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c446e-7a48-46ed-a6a7-5286fc5fbd71 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:50.392814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c446e-7a48-46ed-a6a7-5286fc5fbd71 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:50.393157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c446e-7a48-46ed-a6a7-5286fc5fbd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:50.493339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b74c446e-7a48-46ed-a6a7-5286fc5fbd71 None None] MFA Rules not processed for user `e977eb2e34a945b4adeffbc92d154248`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:50.494463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b74c446e-7a48-46ed-a6a7-5286fc5fbd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:50.547788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:50.603807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 703/1399] 10.222.0.72 () {58 vars in 1040 bytes} [Wed Aug 24 07:48:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:50.613231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:50.614747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:50.751745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d89b81-1199-4a57-bdca-a827b2663712 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:50.822678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:50.856938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:50.857550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:50.857830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:50.858101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:50.859732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:50.943922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:50.945545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9114eb-1a0a-48c1-bfe2-ba90f1d53c10 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:50.953473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 697/1400] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:48:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:48:51.139975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:51.140262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:51.140590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:51.142068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 704/1401] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:51.147519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfa6ac-902a-49ea-a3c7-08a3a8803e8e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:51.147867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfa6ac-902a-49ea-a3c7-08a3a8803e8e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:51.148138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfa6ac-902a-49ea-a3c7-08a3a8803e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:51.220294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d2bfa6ac-902a-49ea-a3c7-08a3a8803e8e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:51.221614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d2bfa6ac-902a-49ea-a3c7-08a3a8803e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:51.286874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:51.334990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 698/1402] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:51.671267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:51.672847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:51.846517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca6c2-9ef0-44a5-a340-938c6d2b5a64 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:51.878037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:51.997507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:51.997875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:51.998112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:51.998382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:52.001490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:52.125692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:52.127839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99105f-a544-4c1d-a229-7d73d327d62a service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:52.152082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 705/1403] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:54.216560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:54.217202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:54.217304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:54.218693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 699/1404] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:54.224357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99245-7e70-4d25-be1f-bb22a2b9dcc9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:54.224847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99245-7e70-4d25-be1f-bb22a2b9dcc9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:54.225109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99245-7e70-4d25-be1f-bb22a2b9dcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:54.272464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:54.304289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-11e99245-7e70-4d25-be1f-bb22a2b9dcc9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:54.305519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-11e99245-7e70-4d25-be1f-bb22a2b9dcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:54.419029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 706/1405] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:54.720366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:54.722531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:54.897173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e241664-ff91-4ae4-b7dc-138b8ede8af1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:54.945562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:55.030937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:55.031556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:55.031847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:55.032108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:55.033766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:55.134815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:55.136492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2f7fc-dfb2-4193-b933-25d208687973 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:55.158478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 700/1406] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:48:57.280736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:57.281299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:57.281299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:57.282693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 707/1407] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:48:57.288824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2cd9-e4e2-4d9a-a4f1-6ba33e26acd8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:57.289638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2cd9-e4e2-4d9a-a4f1-6ba33e26acd8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:57.289910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b2cd9-e4e2-4d9a-a4f1-6ba33e26acd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:57.357800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:57.364918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-401b2cd9-e4e2-4d9a-a4f1-6ba33e26acd8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:48:57.368574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-401b2cd9-e4e2-4d9a-a4f1-6ba33e26acd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:57.485526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 701/1408] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:48:57.784121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:48:57.785917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:57.897989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:57.947676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eb245-386f-45e8-b5ad-518073df8f49 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:48:58.074347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:48:58.075060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:48:58.075344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:48:58.075620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:48:58.077286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:48:58.157764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:48:58.180708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:48:58.182347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11286b82-9f73-4ddd-9eee-6cb948a6f0cb service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:48:58.201846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 708/1409] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:07.481789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=44 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:08.200049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=14 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:15.973621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ef12-d794-4ee6-8323-c82bf8efee7f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:15.974194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ef12-d794-4ee6-8323-c82bf8efee7f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:15.974194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ef12-d794-4ee6-8323-c82bf8efee7f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:15.975487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 702/1410] 10.222.0.72 () {54 vars in 918 bytes} [Wed Aug 24 07:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:15.980529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050b76-24f9-44d9-b8dc-c540c3fa3a35 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:15.980998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050b76-24f9-44d9-b8dc-c540c3fa3a35 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:15.981129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050b76-24f9-44d9-b8dc-c540c3fa3a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:16.057576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-28050b76-24f9-44d9-b8dc-c540c3fa3a35 None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:16.058777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-28050b76-24f9-44d9-b8dc-c540c3fa3a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:16.169353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 709/1411] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Aug 24 07:49:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:16.186737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e83b0-0808-44f6-b1d6-7fc8faad603a None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:16.187020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e83b0-0808-44f6-b1d6-7fc8faad603a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:16.187373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e83b0-0808-44f6-b1d6-7fc8faad603a None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:16.189583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 703/1412] 10.222.0.72 () {54 vars in 918 bytes} [Wed Aug 24 07:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:16.195112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6741ab-df27-49ad-875a-9198ac12d849 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:16.195402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6741ab-df27-49ad-875a-9198ac12d849 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:16.195681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6741ab-df27-49ad-875a-9198ac12d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:16.275126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:16.277581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5a6741ab-df27-49ad-875a-9198ac12d849 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:16.278809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5a6741ab-df27-49ad-875a-9198ac12d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:16.394807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 710/1413] 10.222.0.72 () {58 vars in 1015 bytes} [Wed Aug 24 07:49:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:18.768363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:18.768918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:18.768918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:18.770288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:18.775699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbb65-f1ed-4110-aa78-ec0be0be06d1 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:18.776133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbb65-f1ed-4110-aa78-ec0be0be06d1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:18.776241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbb65-f1ed-4110-aa78-ec0be0be06d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:18.842537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-107fbb65-f1ed-4110-aa78-ec0be0be06d1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:18.843749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-107fbb65-f1ed-4110-aa78-ec0be0be06d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:18.898571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:18.941350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:19.231032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:19.232564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:19.375762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a18e6d-2063-47ae-ab85-76d9a5ca7178 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:19.450427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:19.489697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:19.490285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:19.490561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:19.491023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:19.493291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:19.582336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:19.583943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1773d-6f64-4254-ae81-fecee3f0f725 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:19.601534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 705/1416] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:20.500284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:20.500878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:20.500878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:20.502898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:20.507725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe3027-a063-4bf1-9539-b88d92a96134 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:20.508141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe3027-a063-4bf1-9539-b88d92a96134 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:20.508266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe3027-a063-4bf1-9539-b88d92a96134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:20.581931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-28fe3027-a063-4bf1-9539-b88d92a96134 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:20.583117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-28fe3027-a063-4bf1-9539-b88d92a96134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:20.608971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:20.679895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:20.829737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:20.831324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:20.978768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663557c-cc43-463c-9a7d-c9a314c33ab1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:21.022294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:21.094545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:21.095151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:21.095430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:21.095710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:21.098059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:21.185774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:21.187357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e110dd-bf2c-485c-b2b2-ad4548d1b80f service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:21.205194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 713/1419] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:49:22.089402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:22.089988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:22.089988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:22.091330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:22.096216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a02a-6675-41bc-87e2-f068ba8318a1 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:22.096603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a02a-6675-41bc-87e2-f068ba8318a1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:22.096823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a02a-6675-41bc-87e2-f068ba8318a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:22.151698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:22.172823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0d80a02a-6675-41bc-87e2-f068ba8318a1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:22.174538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d80a02a-6675-41bc-87e2-f068ba8318a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:22.271868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:22.421331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:22.422836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:22.566142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74a766-e02d-4c59-8c43-03469410871b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:22.578179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:22.678348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:22.678863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:22.679135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:22.679402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:22.681011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:22.768516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:22.770218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8f02-7dda-455b-813a-dd66e6eb6b36 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:22.788367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 708/1422] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:23.652785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cfb74-772b-4341-8329-128f75342825 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:23.653357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cfb74-772b-4341-8329-128f75342825 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:23.653357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cfb74-772b-4341-8329-128f75342825 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:23.654750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:23.659684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f734c4-83da-49ca-b3c7-845f996199cc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:23.660160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f734c4-83da-49ca-b3c7-845f996199cc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:23.660260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f734c4-83da-49ca-b3c7-845f996199cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:23.681671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:23.739075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-16f734c4-83da-49ca-b3c7-845f996199cc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:23.740291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-16f734c4-83da-49ca-b3c7-845f996199cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:23.844116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:23.993575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cfb74-772b-4341-8329-128f75342825 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:23.995022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-183cfb74-772b-4341-8329-128f75342825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:24.104151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:24.141764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cfb74-772b-4341-8329-128f75342825 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:24.257733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:24.258246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:24.258527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:24.258802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:24.260412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:24.336349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:24.350230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:24.351829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dfed7-d230-4013-957a-e1666a46a2d0 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:24.369727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 716/1425] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:25.247203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:25.247778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:25.247778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:25.249145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:25.254159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd903-5cc0-434c-86e8-6714bd85eabe None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:25.254598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd903-5cc0-434c-86e8-6714bd85eabe None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:25.254711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd903-5cc0-434c-86e8-6714bd85eabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:25.330238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-75dfd903-5cc0-434c-86e8-6714bd85eabe None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:25.331517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-75dfd903-5cc0-434c-86e8-6714bd85eabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:25.445823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:25.597155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:25.598658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:25.683744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:25.744951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a0ef8-be32-4592-8b32-4454b10b42b5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:25.869422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:25.869941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:25.870221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:25.870495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:25.872232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:25.920393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:25.962447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:25.964060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7da909-4667-4942-92cb-7ea35d1cc226 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:25.982402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 711/1428] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:26.837533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:26.838044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:26.838152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:26.839431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:26.844465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afeed-c468-4d68-a653-51c4078cc48a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:26.844942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afeed-c468-4d68-a653-51c4078cc48a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:26.845158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311afeed-c468-4d68-a653-51c4078cc48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:26.921446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-311afeed-c468-4d68-a653-51c4078cc48a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:26.922669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-311afeed-c468-4d68-a653-51c4078cc48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:27.020008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:27.176005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:27.177839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:27.226850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:27.324740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239dd6-98d1-4afa-84a6-1700f570ccf5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:27.436828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:27.437434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:27.437712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:27.437982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:27.439581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:27.446389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:27.530527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:27.532127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30312-764b-4663-90dd-990d700d5e26 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:27.549785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 719/1431] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:28.396234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:28.396849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:28.396849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:28.398154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:28.404085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b0505-764a-42f0-9d48-2dcde527410b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:28.404573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b0505-764a-42f0-9d48-2dcde527410b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:28.404708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b0505-764a-42f0-9d48-2dcde527410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:28.479689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-735b0505-764a-42f0-9d48-2dcde527410b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:28.480985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-735b0505-764a-42f0-9d48-2dcde527410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:28.547927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:28.576580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:28.728192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:28.729719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:28.747018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:28.874803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bc2a3-a985-4807-a13c-74fdea9e138b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:28.960912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:28.986953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:28.987532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:28.988025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:28.988487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:28.990389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:29.078791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:29.080973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bc9c9-52c8-4ca5-a493-cf83a2e97325 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:29.098531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 714/1434] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:29.983936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:29.984524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:29.984634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:29.985979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:29.991395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e705-8ef0-483d-a54b-14687d6a3785 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:29.991782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e705-8ef0-483d-a54b-14687d6a3785 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:29.991946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609e705-8ef0-483d-a54b-14687d6a3785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:30.077062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0609e705-8ef0-483d-a54b-14687d6a3785 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:30.078328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0609e705-8ef0-483d-a54b-14687d6a3785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:30.129564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:30.185019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:30.336922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:30.338446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:30.488515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199431b-70ab-4817-9a78-495bca777f1f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:30.541780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:30.600540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:30.601104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:30.601389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:30.601666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:30.603303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:30.692630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:30.694243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06fefd-7f4c-4810-9236-27c29af16e45 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:30.713049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 722/1437] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:31.574151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214548-73df-4eee-8ba4-932945071a8c None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:31.574717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214548-73df-4eee-8ba4-932945071a8c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:31.574717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13214548-73df-4eee-8ba4-932945071a8c None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:31.576033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:31.581776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90301-f134-4331-a897-c6235e1b540b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:31.582210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90301-f134-4331-a897-c6235e1b540b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:31.582319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90301-f134-4331-a897-c6235e1b540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:31.657893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-65a90301-f134-4331-a897-c6235e1b540b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:31.659027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-65a90301-f134-4331-a897-c6235e1b540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:31.661066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:31.756839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:31.915851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214548-73df-4eee-8ba4-932945071a8c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:31.917314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-13214548-73df-4eee-8ba4-932945071a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:32.062441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13214548-73df-4eee-8ba4-932945071a8c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:32.088711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:32.176109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:32.176614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:32.177173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:32.177445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:32.179048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:32.266069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:32.267738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ba1ae-6318-4eca-9649-01a712e02f92 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:32.286371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 717/1440] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:34.097760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:34.098306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:34.098306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:34.099625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:34.104901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c34c-bc0c-4bd7-83a7-02f7dd23bbf0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:34.105499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c34c-bc0c-4bd7-83a7-02f7dd23bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:34.105499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c34c-bc0c-4bd7-83a7-02f7dd23bbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:34.146768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:34.172903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cdb8c34c-bc0c-4bd7-83a7-02f7dd23bbf0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:34.174157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cdb8c34c-bc0c-4bd7-83a7-02f7dd23bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:34.273809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:34.559285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:34.560873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:34.663936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773e033-32a4-49b4-9b9c-9c20d3100d64 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:34.696618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:34.738713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:34.740874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:34.740874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:34.740874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:34.741679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:34.851635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:34.853096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d44b-b51f-49b6-af3a-cd7053d8b105 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:34.871961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 725/1443] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:37.028910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:37.029445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:37.029445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:37.030691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:37.036048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb16b6-bf5a-43ca-940b-18d9d9ff0a9f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:37.036448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb16b6-bf5a-43ca-940b-18d9d9ff0a9f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:37.036583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb16b6-bf5a-43ca-940b-18d9d9ff0a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:37.102483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-20eb16b6-bf5a-43ca-940b-18d9d9ff0a9f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:37.103678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-20eb16b6-bf5a-43ca-940b-18d9d9ff0a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:37.116830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:37.199384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:37.485181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:37.487291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:37.589566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620d24e-8c7b-4fc8-84e9-a682bffbbd3c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:37.602326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:37.663760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:37.664280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:37.664557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:37.664857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:37.666443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:37.773860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:37.775277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a24ce9-5480-45e2-aae3-bc5eb393ac00 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:37.793777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 720/1446] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:40.090397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:40.090990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:40.090990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:40.092298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:49:40.097624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ae64-fad8-4978-a561-d4778a52d725 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:40.098033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ae64-fad8-4978-a561-d4778a52d725 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:40.098166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ae64-fad8-4978-a561-d4778a52d725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:40.152862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:40.170467 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6631ae64-fad8-4978-a561-d4778a52d725 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:40.171683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6631ae64-fad8-4978-a561-d4778a52d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:40.267813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:40.551830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:40.553413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:40.654909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6f8b-6987-41cb-a9fb-0db16a3999ff None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:40.695699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:40.729835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:40.730464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:40.730738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:40.731005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:40.732596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:40.848534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:40.849997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b4d78-3e60-4176-b17b-9f401ebc539c service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:40.867642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 728/1449] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:42.898176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:42.898736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:42.898736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:42.900068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:42.905632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857aeba-a2ce-40c6-9bc4-d8c2d7b20d25 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:42.906074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857aeba-a2ce-40c6-9bc4-d8c2d7b20d25 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:42.906187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857aeba-a2ce-40c6-9bc4-d8c2d7b20d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:42.971766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8857aeba-a2ce-40c6-9bc4-d8c2d7b20d25 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:42.972906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8857aeba-a2ce-40c6-9bc4-d8c2d7b20d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:42.993582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:43.068525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:43.364995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:43.368537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:43.480457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a7cf-4fad-43db-8370-ac1f9aa2cda0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:43.503166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:43.560266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:43.560901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:43.561180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:43.561446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:43.563038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:43.669783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:43.671107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6de865-07db-4a05-9c50-5789f92921d0 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:43.690149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 723/1452] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:45.676316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:45.677181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:45.677181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:45.678430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:45.683662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fbbb3-c89e-4c97-9b24-07935cce5229 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:45.684104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fbbb3-c89e-4c97-9b24-07935cce5229 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:45.684202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fbbb3-c89e-4c97-9b24-07935cce5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:45.747199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:45.750117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d85fbbb3-c89e-4c97-9b24-07935cce5229 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:45.751345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d85fbbb3-c89e-4c97-9b24-07935cce5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:45.865892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:46.151070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:46.152624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:46.255797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e7db8-c424-474e-8415-851b71b96787 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:46.304041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:46.328818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:46.330009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:46.330290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:46.330564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:46.332206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:46.564517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:46.565985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48000-206e-4939-9ee8-211f3ed9876c service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:46.584834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 731/1455] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:48.533372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:48.533934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:48.533934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:48.535260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:48.540810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4dd3-78b0-4725-8b90-cb26b1f2b735 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:48.541238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4dd3-78b0-4725-8b90-cb26b1f2b735 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:48.541376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4dd3-78b0-4725-8b90-cb26b1f2b735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:48.605917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5e3d4dd3-78b0-4725-8b90-cb26b1f2b735 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:48.607136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5e3d4dd3-78b0-4725-8b90-cb26b1f2b735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:48.636130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:48.703262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:48.987493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:48.989074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:49.091642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edec09-34ca-4680-8fb2-fbfb9f4bf47c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:49.120388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:49.168083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:49.168715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:49.168996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:49.169267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:49.170859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:49.398122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:49.399577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82001f15-0191-4acf-9600-2ad7358bb5d8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:49.417657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 726/1458] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:51.409275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:51.409779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:51.409865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:51.411246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:51.420450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae8c5-95d2-4716-9e8f-15a2737f3781 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:51.421013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae8c5-95d2-4716-9e8f-15a2737f3781 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:51.421129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae8c5-95d2-4716-9e8f-15a2737f3781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:51.490463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-313ae8c5-95d2-4716-9e8f-15a2737f3781 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:51.491681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-313ae8c5-95d2-4716-9e8f-15a2737f3781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:51.502048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:51.611239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:51.919495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:51.921102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:52.048446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd70841-66bd-430d-8bde-84fdbbd0b57f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:52.104491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:52.122348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:52.122950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:52.123223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:52.123485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:52.125181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:52.233759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:52.235095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32437-1cd9-4f99-9666-37d5e1ec7e25 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:52.254500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 734/1461] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:54.212446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.213038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.213038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.214281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:54.219763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e112-5374-4a75-9e28-1d3de8f88d57 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.220176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e112-5374-4a75-9e28-1d3de8f88d57 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.220343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50e112-5374-4a75-9e28-1d3de8f88d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.290247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5b50e112-5374-4a75-9e28-1d3de8f88d57 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:54.291463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5b50e112-5374-4a75-9e28-1d3de8f88d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:54.330130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:54.388271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:49:54.669621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:54.671152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:54.776152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0295e9-fff5-4404-b476-18fbaea0e08f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:54.813760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:54.850866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:54.851466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:54.851741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:54.852036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:54.853648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:54.961070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:54.962487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6cbd7-09e5-4a6a-80ec-e524f4825377 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:54.981192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 729/1464] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:56.900065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:56.900718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:56.900718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:56.902018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:49:56.907417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb30c-fbf0-400a-b507-b18f7c60fe1d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:56.907867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb30c-fbf0-400a-b507-b18f7c60fe1d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:56.908015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb30c-fbf0-400a-b507-b18f7c60fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:56.975249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-861bb30c-fbf0-400a-b507-b18f7c60fe1d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:56.976412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-861bb30c-fbf0-400a-b507-b18f7c60fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:56.995098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:57.075929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:49:57.358976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:49:57.361065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:57.464632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60750637-bd14-42d4-ad3b-d68df167d462 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:49:57.534338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:57.541249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:49:57.550060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:57.550060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:57.550060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:57.550060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:57.651874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:49:57.653269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069931-b083-4e74-bcb6-1dd7e2b238eb service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:49:57.671272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 737/1467] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:49:59.604278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:59.604877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:59.604877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:59.606189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:49:59.611882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8c11-3916-458f-a6fc-bda633923748 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:49:59.612272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8c11-3916-458f-a6fc-bda633923748 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:49:59.612442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8c11-3916-458f-a6fc-bda633923748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:49:59.679553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4e7e8c11-3916-458f-a6fc-bda633923748 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:49:59.680736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4e7e8c11-3916-458f-a6fc-bda633923748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:49:59.731720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:49:59.779034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:00.062565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:00.064267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:00.166748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc583f-9b50-4407-a382-82d3ba3c6695 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:00.213347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:00.239565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:00.240168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:00.240445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:00.240741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:00.242330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:00.351330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:00.352720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8aa48-de0f-4d30-bb13-43bf83a8cc17 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:00.370938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 732/1470] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:09.804883 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=22 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:10.368716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=66 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:19.960095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239de7-654c-49d0-a2b9-d5dba40b9fa2 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:19.960682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239de7-654c-49d0-a2b9-d5dba40b9fa2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:19.960682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6239de7-654c-49d0-a2b9-d5dba40b9fa2 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:19.962016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 739/1471] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:19.967475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10afd8-fed7-4bb1-bd4a-d73e62e3ebc4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:19.967935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10afd8-fed7-4bb1-bd4a-d73e62e3ebc4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:19.968068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10afd8-fed7-4bb1-bd4a-d73e62e3ebc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:20.033474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7b10afd8-fed7-4bb1-bd4a-d73e62e3ebc4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:20.034688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7b10afd8-fed7-4bb1-bd4a-d73e62e3ebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:20.133162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 733/1472] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:20.427971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:20.428252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:20.428551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:20.429938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 740/1473] 10.222.0.72 () {54 vars in 967 bytes} [Wed Aug 24 07:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:20.434948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca67-e496-4fc7-a2fc-eb2b6acf904e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:20.435238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca67-e496-4fc7-a2fc-eb2b6acf904e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:20.435515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca67-e496-4fc7-a2fc-eb2b6acf904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:20.510061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-030aca67-e496-4fc7-a2fc-eb2b6acf904e None None] MFA Rules not processed for user `b95e296a3e2f48599b916a0ac7be2213`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:20.511263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-030aca67-e496-4fc7-a2fc-eb2b6acf904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:20.532527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:20.586964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 734/1474] 10.222.0.72 () {58 vars in 1064 bytes} [Wed Aug 24 07:50:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:20.594994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:20.596616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:20.707641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b2983-1dec-4313-95ea-fcde3beac5a6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:20.782330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:20.782930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:20.783216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:20.783472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:20.785062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:20.838145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:20.891945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:20.893387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567168e9-121c-485c-9258-5638b3e40899 service glance] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:20.911525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 741/1475] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:50:22.906580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:22.907120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:22.907120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:22.908464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 735/1476] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:22.913673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a84c1-fbf0-4b60-988d-e6966f023bb8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:22.914122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a84c1-fbf0-4b60-988d-e6966f023bb8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:22.914317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a84c1-fbf0-4b60-988d-e6966f023bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:22.981417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2a6a84c1-fbf0-4b60-988d-e6966f023bb8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:22.982631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2a6a84c1-fbf0-4b60-988d-e6966f023bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:23.080994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 742/1477] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:50:23.358377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:23.359871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:23.486068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6405c7a-bbf3-4506-a013-10c12fedd435 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:23.533805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:23.578284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:23.578879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:23.579150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:23.579415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:23.580037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:23.581798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:50:23.581798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:50:23.581798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:50:23.582926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03090ce-009c-489f-95dc-621e4a1e6fb1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:23.600976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 736/1478] 10.222.0.72 () {56 vars in 1108 bytes} [Wed Aug 24 07:50:23 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:50:25.541107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:25.541676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:25.541676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:25.543026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 743/1479] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:25.548936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5a16-7637-42d9-8ecc-4b2bca3e76f4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:25.549370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5a16-7637-42d9-8ecc-4b2bca3e76f4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:25.549505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc5a16-7637-42d9-8ecc-4b2bca3e76f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:25.615425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a1dc5a16-7637-42d9-8ecc-4b2bca3e76f4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:25.616638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a1dc5a16-7637-42d9-8ecc-4b2bca3e76f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:25.712467 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 737/1480] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:26.008915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:26.010517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:26.020756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:26.168190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfb395-1e23-4a96-b091-d47f35374f7f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:26.246062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:26.280263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:26.280895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:26.281174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:26.281444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:26.283037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:26.373806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:26.376232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08de08c-2572-464b-8ec5-470acfec2ee3 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:26.396530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 744/1481] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:28.511868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:28.512487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:28.512487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:28.513903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 738/1482] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 07:50:28.519384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2597ab-5822-4c1f-8786-3843713bc936 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:28.519808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2597ab-5822-4c1f-8786-3843713bc936 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:28.519909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2597ab-5822-4c1f-8786-3843713bc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:28.586451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-cd2597ab-5822-4c1f-8786-3843713bc936 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:28.587662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-cd2597ab-5822-4c1f-8786-3843713bc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:28.633001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:28.683637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 745/1483] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:28.970752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:28.972263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:28.995122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:29.074580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772b588-8263-4260-9b3f-b08eaa2ff547 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:29.148356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:29.148991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:29.149272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:29.149544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:29.151152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:29.226702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:29.260507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:29.262252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054bfc0-ef82-4e0b-964f-c2a5e0be1a9e service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:29.280641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 739/1484] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:31.267247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:31.267821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:31.267821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:31.269148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 746/1485] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:31.274360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae509e9-0ea0-45e3-8fd1-89bc1b289544 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:31.274793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae509e9-0ea0-45e3-8fd1-89bc1b289544 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:31.274937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae509e9-0ea0-45e3-8fd1-89bc1b289544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:31.340485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dae509e9-0ea0-45e3-8fd1-89bc1b289544 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:31.341722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dae509e9-0ea0-45e3-8fd1-89bc1b289544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:31.443222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 740/1486] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:31.749783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:31.751364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:31.858579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50a92f-f1ae-4164-b9f7-6f31ab0d86f6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:31.929809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:31.931688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:31.932291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:31.932571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:31.932943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:31.934544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:32.041591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:32.042969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c59fa-eb2a-4957-8331-bc27bfe62364 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:32.062485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 747/1487] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:34.122773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:34.123360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:34.123360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:34.124856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 741/1488] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:34.129988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61586084-90bd-4c58-875d-8a95aceacd74 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:34.130416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61586084-90bd-4c58-875d-8a95aceacd74 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:34.130542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61586084-90bd-4c58-875d-8a95aceacd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:34.204745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-61586084-90bd-4c58-875d-8a95aceacd74 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:34.205929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-61586084-90bd-4c58-875d-8a95aceacd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:34.258889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:34.302772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 748/1489] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:34.590583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:34.592128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:34.628418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:34.696679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea479f-49e7-4f60-9b7a-55add859b663 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:34.770017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:34.770622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:34.770898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:34.771165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:34.772786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:34.850804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:34.882171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:34.883672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25baa42-e110-454b-ace0-2023b9719455 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:34.902143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 742/1490] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:37.157972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:37.158538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:37.159088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:37.162036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 749/1491] 10.222.0.72 () {54 vars in 927 bytes} [Wed Aug 24 07:50:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:50:37.167357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20579a-7623-4694-804f-e6c4a8b1fcf7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:37.167763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20579a-7623-4694-804f-e6c4a8b1fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:37.167905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20579a-7623-4694-804f-e6c4a8b1fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:37.234074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fa20579a-7623-4694-804f-e6c4a8b1fcf7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:50:37.235287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fa20579a-7623-4694-804f-e6c4a8b1fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:37.332235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 743/1492] 10.222.0.72 () {58 vars in 1024 bytes} [Wed Aug 24 07:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:50:37.619705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:50:37.621314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:37.726961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eade4-d6e8-4833-917a-47a2afaf0589 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:50:37.811365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:50:37.812564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:50:37.812869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:50:37.813139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:50:37.814727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:50:37.823688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:37.923114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:50:37.924476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971bfbb-74c8-48c1-aab3-4f4c77a511f9 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:50:37.943669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 750/1493] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:50:47.331016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=82 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:50:47.941743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=50 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:57.908011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:57.908639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:57.908639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:57.982658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:57.983905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.080441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 744/1494] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:58.088493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:58.088995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:58.089104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:58.166811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:58.168438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.265806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 751/1495] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:58.270554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:58.271799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.283063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c59db7-5bf9-4c72-8e46-2083c6c6c687 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:58.343743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:58.377414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:58.378018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:58.378291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:58.378557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:58.379285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:51:58.379832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:58.380600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:58.380600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:51:58.380600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:51:58.381511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:58.393450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 745/1496] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:51:58.401931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:58.411141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c055f12-6c7a-44f2-b7ad-572aece3e5aa None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:58.471261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:58.505501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:58.506105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:58.506403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:58.506660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:58.507291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:58.509037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:58.509037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:51:58.509037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:51:58.510221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:58.645926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 752/1497] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:51:58 2022] POST /identity/v3/projects => generated 377 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:58.650378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:58.651912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.686900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9509-2930-49eb-82f1-fe5b43bc057e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:58.782504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:58.783006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:58.783285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:58.783555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:58.784187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:58.785223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:58.785223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:51:58.785223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:51:58.786435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:58.876482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 746/1498] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:51:58 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:51:58.880774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:58.882447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:58.945048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852c2ad-4cb3-43ad-852e-b80cedb3406d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:59.026564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:59.041711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:59.041711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.041711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.041711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.042137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:59.042137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:59.042137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:51:59.042137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:51:59.042137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:59.053979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 753/1499] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:51:59.058109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:59.059630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.071002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0f78a-3920-4ba1-9769-f79e5cc012ab admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:59.096797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:59.167742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:59.168959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.169260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.169527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.170175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:59.171187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:59.171187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:51:59.171187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:51:59.172073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58732742-df41-47e4-befb-d78a54c31846 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:59.184534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 747/1500] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:51:59.188770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:59.190304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.223514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e03c9a-3941-45a9-8615-73843e0502ce admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:59.317506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:59.318169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.318450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.318722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] PATH_INFO: `/v3/projects/74561bb847f14caa84bc6e26cdb0075e/users/6ec0686af4034091bc113d1429e78b9a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.340436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74561bb847f14caa84bc6e26cdb0075e, user_id=6ec0686af4034091bc113d1429e78b9a, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:51:59.341500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:51:59.341500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:51:59.341500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:51:59.343444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300a5c0-1035-4766-a14c-6ffffbceac9a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:51:59.442240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 754/1501] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:51:59 2022] PUT /identity/v3/projects/74561bb847f14caa84bc6e26cdb0075e/users/6ec0686af4034091bc113d1429e78b9a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:51:59.455106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a367e7-343b-4f5a-a441-4649f83a607e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.455396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a367e7-343b-4f5a-a441-4649f83a607e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.455674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a367e7-343b-4f5a-a441-4649f83a607e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.513147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a9a367e7-343b-4f5a-a441-4649f83a607e None None] MFA Rules not processed for user `6ec0686af4034091bc113d1429e78b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:59.514456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a9a367e7-343b-4f5a-a441-4649f83a607e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.591098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 748/1502] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:51:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:51:59.604134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.604421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.604724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.606073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 755/1503] 10.222.0.72 () {54 vars in 964 bytes} [Wed Aug 24 07:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:51:59.610845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2102-1eb1-48e8-938b-e01ca0de7f1c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.611129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2102-1eb1-48e8-938b-e01ca0de7f1c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.611399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc2102-1eb1-48e8-938b-e01ca0de7f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.641749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:59.687697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2fbc2102-1eb1-48e8-938b-e01ca0de7f1c None None] MFA Rules not processed for user `8a2fd7d0bbcb4a04861d7cf060c671a8`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:51:59.689324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2fbc2102-1eb1-48e8-938b-e01ca0de7f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.766191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 749/1504] 10.222.0.72 () {58 vars in 1061 bytes} [Wed Aug 24 07:51:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:51:59.774706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:51:59.776235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:51:59.835711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:51:59.894499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83196ddd-b30c-4282-8103-bb853bed1f97 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:51:59.969778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:51:59.970400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:51:59.970735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:51:59.971009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:51:59.972794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:00.073119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:00.082107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:00.092130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 756/1505] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:51:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:00.137430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8a6a-f3c4-49b5-a5e0-d28bae6a9c0b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.137906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8a6a-f3c4-49b5-a5e0-d28bae6a9c0b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.138006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8a6a-f3c4-49b5-a5e0-d28bae6a9c0b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.139061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:00.139444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 750/1506] 10.222.0.72 () {52 vars in 830 bytes} [Wed Aug 24 07:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:52:00.142802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:00.144149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:00.243844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd409d-b969-4bf8-9730-61330a28d79c service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:00.336297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:00.336953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.337232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.337507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] PATH_INFO: `/v3/users/6ec0686af4034091bc113d1429e78b9a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.349618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec0686af4034091bc113d1429e78b9a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:00.350685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:52:00.350685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:52:00.350685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:52:00.351900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:00.362053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:00.617415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec0686af4034091bc113d1429e78b9a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.631381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec0686af4034091bc113d1429e78b9a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.631938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec0686af4034091bc113d1429e78b9a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.632362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec0686af4034091bc113d1429e78b9a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.715456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec0686af4034091bc113d1429e78b9a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:00.747071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 757/1507] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:52:00 2022] DELETE /identity/v3/users/6ec0686af4034091bc113d1429e78b9a => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:52:00.751562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:00.751849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:00.752124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:00.810875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:52:00.812095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:00.888035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:00.900336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 751/1508] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:52:00.910631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:00.911875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.010902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f81a-f6d3-423d-872e-6b672777618e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.066102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:01.085108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.085726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.086007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.086282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.087884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.197201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.198572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9faee-5883-432b-acaa-bdd0ae3010d8 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:01.217391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 758/1509] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:52:01.728460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:52:01.729807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:52:01.741753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028cac8-237e-4fa8-9a1b-68706c73e4eb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:52:01.841045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:52:01.841681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:52:01.841959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:52:01.842230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] PATH_INFO: `/v3/projects/74561bb847f14caa84bc6e26cdb0075e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:52:01.852145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74561bb847f14caa84bc6e26cdb0075e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:52:01.853208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:52:01.853208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:52:01.853208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:52:01.854417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:52:02.051020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74561bb847f14caa84bc6e26cdb0075e', 'request_id': 'req-b5e786de-cc88-4b75-b608-ae54dba8ef59'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:02.149674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74561bb847f14caa84bc6e26cdb0075e', 'request_id': 'req-b5e786de-cc88-4b75-b608-ae54dba8ef59'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:02.163840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74561bb847f14caa84bc6e26cdb0075e', 'request_id': 'req-b5e786de-cc88-4b75-b608-ae54dba8ef59'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:02.188772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] The token cache is being invalidate because project 74561bb847f14caa84bc6e26cdb0075e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:52:02.189061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:02.189495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5e786de-cc88-4b75-b608-ae54dba8ef59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:52:02.190880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 752/1510] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:52:01 2022] DELETE /identity/v3/projects/74561bb847f14caa84bc6e26cdb0075e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:52:11.215189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=62 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:52:12.186706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:30.333320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.333923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.333923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.335276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:30.340452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a08e4-738c-4feb-8e19-28bf4d7ec05f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.344460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a08e4-738c-4feb-8e19-28bf4d7ec05f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.344460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a08e4-738c-4feb-8e19-28bf4d7ec05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.422955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3a6a08e4-738c-4feb-8e19-28bf4d7ec05f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:30.424234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3a6a08e4-738c-4feb-8e19-28bf4d7ec05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:30.528084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:30.684357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:30.687148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:30.806760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c0cc6-a62e-41fc-9954-0652c8889561 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:30.812923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:30.813532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:30.813812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:30.814085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:30.823539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:30.825170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2af738-2d32-43e3-bc6c-bb7febb006e2 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:30.834494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 760/1513] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:31.673739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a188-c7df-4fba-9be7-ab4c999ce2f5 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.674273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a188-c7df-4fba-9be7-ab4c999ce2f5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.674273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a188-c7df-4fba-9be7-ab4c999ce2f5 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.675553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:31.680449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2f98-ef8e-41e5-bec7-6459295983f7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.680895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2f98-ef8e-41e5-bec7-6459295983f7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.681035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2f98-ef8e-41e5-bec7-6459295983f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.700441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-909e2f98-ef8e-41e5-bec7-6459295983f7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:31.702197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-909e2f98-ef8e-41e5-bec7-6459295983f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:31.710187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:31.863459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a188-c7df-4fba-9be7-ab4c999ce2f5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:31.878703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a188-c7df-4fba-9be7-ab4c999ce2f5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:31.884214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:31.884844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:31.885124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:31.885399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:31.894347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:31.895931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd77bc-74ac-4928-b4a4-da41ff6b63ac service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:31.905694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 755/1516] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:32.499425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1cb-a336-43de-8b3e-2d610776ba21 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:32.499710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1cb-a336-43de-8b3e-2d610776ba21 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:32.499981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1b1cb-a336-43de-8b3e-2d610776ba21 None None] PATH_INFO: `/v3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:32.501311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 762/1517] 10.222.0.45 () {54 vars in 925 bytes} [Wed Aug 24 07:53:32 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:53:32.507212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96494a-be3e-497b-8a29-b945676375de None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:32.507500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96494a-be3e-497b-8a29-b945676375de None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:32.507775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96494a-be3e-497b-8a29-b945676375de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:32.567800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ab96494a-be3e-497b-8a29-b945676375de None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:32.569256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ab96494a-be3e-497b-8a29-b945676375de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:32.586203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:32.640347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 756/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 07:53:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:36.692978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2604-ad6d-47f8-a886-ee12aec611c8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.693574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2604-ad6d-47f8-a886-ee12aec611c8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.693574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2604-ad6d-47f8-a886-ee12aec611c8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.694863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 763/1519] 10.222.0.72 () {54 vars in 961 bytes} [Wed Aug 24 07:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:36.700200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da06d-c245-4482-a7dd-b601afaee964 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.700621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da06d-c245-4482-a7dd-b601afaee964 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.700804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42da06d-c245-4482-a7dd-b601afaee964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.720015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d42da06d-c245-4482-a7dd-b601afaee964 None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:36.721251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d42da06d-c245-4482-a7dd-b601afaee964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:36.729754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 757/1520] 10.222.0.72 () {58 vars in 1058 bytes} [Wed Aug 24 07:53:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:36.737839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2604-ad6d-47f8-a886-ee12aec611c8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:36.751659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2604-ad6d-47f8-a886-ee12aec611c8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:36.756702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:36.757294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:36.757578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:36.757852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:36.766121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:36.767699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55333c1-a81d-4447-8a11-98f366ac8672 service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:36.777350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 764/1521] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:43.149736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4395-3705-4afc-b385-1f2350e98241 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:43.150322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4395-3705-4afc-b385-1f2350e98241 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:43.150322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb4395-3705-4afc-b385-1f2350e98241 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:43.151542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:43.156444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463edd45-e83a-4d00-94d7-904506842ca9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:43.156882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463edd45-e83a-4d00-94d7-904506842ca9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:43.157035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463edd45-e83a-4d00-94d7-904506842ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:43.176252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-463edd45-e83a-4d00-94d7-904506842ca9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:43.177475 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-463edd45-e83a-4d00-94d7-904506842ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:43.185805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:43.370547 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb4395-3705-4afc-b385-1f2350e98241 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:43.387081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb4395-3705-4afc-b385-1f2350e98241 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:43.393453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:43.394149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:43.394450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:43.394737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:43.404581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:43.406449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf037b-5105-4607-b967-f40bac3a9ada service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:43.416884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 759/1524] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:44.237049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7335c-3b2c-47fc-bc7f-f0117c994a03 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.237751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7335c-3b2c-47fc-bc7f-f0117c994a03 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.237751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7335c-3b2c-47fc-bc7f-f0117c994a03 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.238968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 07:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:44.244355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef11ff-3b48-4f36-823a-81f5cec0d4fe None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.244846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef11ff-3b48-4f36-823a-81f5cec0d4fe None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.245067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef11ff-3b48-4f36-823a-81f5cec0d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.266108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-36ef11ff-3b48-4f36-823a-81f5cec0d4fe None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:44.267562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-36ef11ff-3b48-4f36-823a-81f5cec0d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:44.276671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 07:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:44.430069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7335c-3b2c-47fc-bc7f-f0117c994a03 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:44.445433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7335c-3b2c-47fc-bc7f-f0117c994a03 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:44.451651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:44.452251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:44.452529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:44.452835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:44.461387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:44.462973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6aeb1-a134-4e8a-a536-cf8b7f823e13 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:44.473277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 767/1527] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:48.428883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:48.429628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:48.429628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:48.430867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:48.437198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329256c3-d47f-40da-87c4-dfb6510fa8e9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:48.437742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329256c3-d47f-40da-87c4-dfb6510fa8e9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:48.437742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329256c3-d47f-40da-87c4-dfb6510fa8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:48.457802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-329256c3-d47f-40da-87c4-dfb6510fa8e9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:48.467323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-329256c3-d47f-40da-87c4-dfb6510fa8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:48.467323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:48.765316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:48.767231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:48.847141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d6da-3f75-4861-bc0f-d7f4941be492 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:48.850742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:48.851372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:48.851650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:48.851921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:48.860878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:48.862298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a812cfe-2700-4fdf-acb7-5584417e6f7e service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:48.871656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 762/1530] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:50.795791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0bb06-b98e-4503-bd57-9bef096c0792 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:50.796323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0bb06-b98e-4503-bd57-9bef096c0792 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:50.796323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0bb06-b98e-4503-bd57-9bef096c0792 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:50.797632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:53:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:50.803070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349c6-ed97-4eeb-b405-eeca5ffa3cec None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:50.803495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349c6-ed97-4eeb-b405-eeca5ffa3cec None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:50.803616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6349c6-ed97-4eeb-b405-eeca5ffa3cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:50.822805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dc6349c6-ed97-4eeb-b405-eeca5ffa3cec None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:50.824013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dc6349c6-ed97-4eeb-b405-eeca5ffa3cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:50.832002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:51.124197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0bb06-b98e-4503-bd57-9bef096c0792 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:51.135044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0bb06-b98e-4503-bd57-9bef096c0792 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:51.138786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:51.139388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:51.139668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:51.139938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:51.149257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:51.150672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5793-72d7-4683-8562-05c9952de9cd service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:51.161962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 770/1533] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:54.640132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:54.640745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:54.640745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:54.642016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:54.647624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17a9-2fb8-4c0b-ae52-40ee8ac83e01 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:54.648036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17a9-2fb8-4c0b-ae52-40ee8ac83e01 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:54.648227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17a9-2fb8-4c0b-ae52-40ee8ac83e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:54.656095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=81 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:54.669914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d69c17a9-2fb8-4c0b-ae52-40ee8ac83e01 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:54.671141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d69c17a9-2fb8-4c0b-ae52-40ee8ac83e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:54.679068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:53:55.259554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:55.261366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:55.331333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:53:55.344363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d09b9-1510-44bb-97ec-2669a9ee6a3a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:55.348023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:55.348627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:55.348931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:55.349201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:55.357969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:55.359479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71f992-b022-4691-81a8-cc92508a23c4 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:55.369003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 765/1536] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:53:57.309955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:57.310667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:57.310667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:57.311977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 07:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:53:57.317375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e467b89-61df-4ea2-af5f-7403283e939c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:57.317813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e467b89-61df-4ea2-af5f-7403283e939c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:57.317923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e467b89-61df-4ea2-af5f-7403283e939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:57.338301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8e467b89-61df-4ea2-af5f-7403283e939c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:53:57.339503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8e467b89-61df-4ea2-af5f-7403283e939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:57.347532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 07:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:53:57.911528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:53:57.913316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:53:57.926394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a11b3-0a90-43ae-8f00-aca62f7b20dc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:53:57.929908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:53:57.930499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:53:57.930776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:53:57.931081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:53:57.939905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:53:57.941369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8f271-c2e7-4df1-91d3-bfc4dbf88332 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:53:57.951267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 773/1539] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:54:07.333623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=11 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:54:07.946604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=8 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:20.721282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:20.723821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:20.723821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:20.747614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:20.748881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:20.764487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 767/1540] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:20.773371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:20.774562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:20.774562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:20.801729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:20.802954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:20.814417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 774/1541] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:55:20.819038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:20.829691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d8c36-dad3-4e67-96f7-0e7b1d653210 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:20.837290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:20.837918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:20.838195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:20.838466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:20.839129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:20.839435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:20.841660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e171822-1220-4baf-b0e8-7b89535152d5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:20.862176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 768/1542] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:20.866873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:20.877312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18495b-fe66-4aab-9485-e566b9d56602 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:20.885235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:20.885898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:20.886161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:20.886435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:20.887095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:20.889613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:20.965314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e5ca0-aa37-4041-b0ea-1535de4c14a0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:20.965732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e5ca0-aa37-4041-b0ea-1535de4c14a0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:20.965848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e5ca0-aa37-4041-b0ea-1535de4c14a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:20.993116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f78e5ca0-aa37-4041-b0ea-1535de4c14a0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:20.994270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f78e5ca0-aa37-4041-b0ea-1535de4c14a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.016089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 775/1543] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:20 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:21.016864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 769/1544] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 07:55:21.020873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.027682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f546e-080d-4370-b6ba-c36a162c8711 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.028117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f546e-080d-4370-b6ba-c36a162c8711 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.028280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f546e-080d-4370-b6ba-c36a162c8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.056374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-984f546e-080d-4370-b6ba-c36a162c8711 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.057634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-984f546e-080d-4370-b6ba-c36a162c8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.137057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 770/1545] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.139047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114c6eb-870b-4cb3-8fc9-a9dd21e310da None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.139333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114c6eb-870b-4cb3-8fc9-a9dd21e310da None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.139607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114c6eb-870b-4cb3-8fc9-a9dd21e310da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.149111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2093-a026-4eab-b8cc-efb532407dce admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.166458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7114c6eb-870b-4cb3-8fc9-a9dd21e310da None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.167058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.167700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.167829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7114c6eb-870b-4cb3-8fc9-a9dd21e310da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.168008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.168478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.169539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.169950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:21.169950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:55:21.169950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:55:21.171160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5afaf-bb5f-4a8f-8613-58806b1f136e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.186975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 771/1546] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.188361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.188644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.188942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.212639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.215010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.222340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 772/1547] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.223150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.232955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0657c-c224-402c-ae27-8d032ee52bfe None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.237629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.238226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.238515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.238786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.239430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.239733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.241464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf2617-0b30-449b-b81e-77f7c1b46414 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.253615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 773/1548] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.254702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6e8f-9ace-48ac-bd6b-c0dd02673f18 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.255009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6e8f-9ace-48ac-bd6b-c0dd02673f18 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.255280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6e8f-9ace-48ac-bd6b-c0dd02673f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.274423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-442c6e8f-9ace-48ac-bd6b-c0dd02673f18 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.275615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-442c6e8f-9ace-48ac-bd6b-c0dd02673f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.283525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 774/1549] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.286229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b237-a72c-4880-8908-379a526d8abb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.289709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b237-a72c-4880-8908-379a526d8abb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.289709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b237-a72c-4880-8908-379a526d8abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.292730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 776/1550] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/users => generated 285 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:21.296817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1b167-b3a8-4a95-a5a1-e2953b05c623 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.297176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1b167-b3a8-4a95-a5a1-e2953b05c623 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.297520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1b167-b3a8-4a95-a5a1-e2953b05c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.307657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5f47b237-a72c-4880-8908-379a526d8abb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.308895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5f47b237-a72c-4880-8908-379a526d8abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.317241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3fc1b167-b3a8-4a95-a5a1-e2953b05c623 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.321347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 775/1551] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.321482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3fc1b167-b3a8-4a95-a5a1-e2953b05c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.321597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6f1a-cf72-45e6-b218-aa56a3428c15 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.321597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6f1a-cf72-45e6-b218-aa56a3428c15 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.321597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6f1a-cf72-45e6-b218-aa56a3428c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.327513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 777/1552] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.329116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.329116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.329440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.340995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-64dc6f1a-cf72-45e6-b218-aa56a3428c15 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.342225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-64dc6f1a-cf72-45e6-b218-aa56a3428c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.348690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.349850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.350230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 776/1553] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.351309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1362e75-c994-4ef5-96b3-8cf321efcea2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.351591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1362e75-c994-4ef5-96b3-8cf321efcea2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.351862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1362e75-c994-4ef5-96b3-8cf321efcea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.363145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 778/1554] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.363960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.373876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419b91b-5407-4fcb-93b7-c8153f75bcdc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.374898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e1362e75-c994-4ef5-96b3-8cf321efcea2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.376129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e1362e75-c994-4ef5-96b3-8cf321efcea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.378604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.379205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.379485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.379753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.380385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.382310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb010-b6d9-4546-af93-f56a36b426cb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.387497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 777/1555] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.389612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.389897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.390173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.411886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.415378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.421352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 778/1556] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.422086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.431170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33849a-5bc0-452b-8d5b-2946c989d3b2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.436223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.436903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.443979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.443979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.443979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.443979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.449830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 779/1557] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:21.451236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.460953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd987e5-4e9b-4c7d-95a3-a751e60de4b9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.465622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.466238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.466524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.466787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.467423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.467727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.469384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad85f41-8588-448a-8547-bfa971cb780c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.481468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 780/1558] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.483181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.483464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.483732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.500558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 779/1559] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:21.503830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.504348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.515539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fe6-1c6d-411b-9f2b-4b021663dad5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.515832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fe6-1c6d-411b-9f2b-4b021663dad5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.516110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fe6-1c6d-411b-9f2b-4b021663dad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.536660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-eedb0fe6-1c6d-411b-9f2b-4b021663dad5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.537890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-eedb0fe6-1c6d-411b-9f2b-4b021663dad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.554625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:21.570230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 781/1560] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.571169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.580772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029fb9-221f-4c82-817c-23946e90c2ee None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.585827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.586446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.586815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.587242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.587883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.588245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.590063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.598752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 780/1561] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.602733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.602733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.602733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.602998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 782/1562] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.609706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.619504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a9746-fbab-4805-9519-bcc55fff2227 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.625027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.625714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.626077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.626378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:21.626492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.627249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.627572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:21.627688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.631326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.638711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 781/1563] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:21.640166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.644400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 783/1564] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.645233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.655056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a62af8-91a7-4d3e-af00-2b4f0a2b4335 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.656382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3857-43cb-4b5d-9d57-6b64c0fc9621 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.660864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.661233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.661559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.661784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.661923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.662086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.662202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.662349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.662766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.662983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.663816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:21.663816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:55:21.663816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:55:21.664870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.667840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.671477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:21.676471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 782/1565] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:21.677370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.689994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4543-88e8-4e45-b263-7c5bef08a643 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.697277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.698287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.698794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.699302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.700763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.704702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.786644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 784/1566] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:21.786644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.816857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 783/1567] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:21.817820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.829101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc678da1-39ed-42c4-bba1-fae18dfd3a7c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.850199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3598bb6-7575-4986-8521-5949a0419134 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.854995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.855609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.855896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.856166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.857345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.857648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.859255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.870630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 785/1568] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.871568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.887735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.888139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fcd4f-f741-415e-a842-609716c82623 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.888354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.888631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.888934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.889594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.889897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.890955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:21.890955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:55:21.890955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:55:21.891872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.892743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.893332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.893794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.893913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.894603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.896779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.904158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 784/1569] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.905042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.916419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49615102-7bd4-4a47-91ff-4bbbc0410f65 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.921779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.922377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.922653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.922927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.923590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:21.923870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.925496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:21.939817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 785/1570] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:21.940621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:21.950506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba9d8e-17e0-4995-a253-378aaa3c7e78 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:21.955285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:21.955904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:21.956179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:21.956446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:21.957147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:21.959015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.000384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 786/1571] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.001250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.059525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 786/1572] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:21 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.061585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.061993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01b579-afb2-4e0a-b5b7-8888d40c05d2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.081196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:22.120190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.120910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.121633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.122025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] PATH_INFO: `/v3/projects/7f6d7dd049f440828cec4efd5d4ef101/users/0d774440fd2141f2b9407520e0cbc94f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.125491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea900a2-cc4d-4c8b-913c-39dd788c9b72 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.130178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.130779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.131053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.131322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.131948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.133907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.136520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f6d7dd049f440828cec4efd5d4ef101, user_id=0d774440fd2141f2b9407520e0cbc94f, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.137598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:22.137598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:55:22.137598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:55:22.139542 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.206149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 787/1573] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:22 2022] PUT /identity/v3/projects/7f6d7dd049f440828cec4efd5d4ef101/users/0d774440fd2141f2b9407520e0cbc94f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:22.206975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.242960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 787/1574] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.242960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.254228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80412801-344b-4158-b3f5-a3d3cfb98a9a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.271091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc738a8-e8a7-46e8-a66e-52b8a0fff21f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.276774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.277433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.277706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.277973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.279333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.280974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.284408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:22.291343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 788/1575] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:22.293533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.304916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d30798-0308-475f-8f76-c3dc525eb680 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.309661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.310189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.310466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.310741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.311399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.312033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.312797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.313110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.313276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.313389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.314029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.315947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.396118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 789/1576] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.397192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.424556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 788/1577] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/projects => generated 371 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.425801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.462946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d59cd-b3ed-48e4-883b-632aa03a9a7f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.487635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:22.488911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8245-87aa-418c-8ff5-cac96db9f260 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.493518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.494127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.494405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.494776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.495346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.497347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.520290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.520939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.521220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.521493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.522116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.524115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93711eb-84d8-4be6-bccc-ad1d5bcdd40e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.654858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 789/1578] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.654858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.662477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aa559-c2b6-492f-9b84-6953e9d17cb6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.666415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 790/1579] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/projects => generated 379 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.667676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.667961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.668234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.670816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.671457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.671741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.672013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.672637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.674694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.696026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] MFA Rules not processed for user `0d774440fd2141f2b9407520e0cbc94f`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:22.697294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:22.759039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 791/1580] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:22.759925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.788972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 790/1581] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:22.790317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.801461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529e403-8bf9-4afd-bc6d-1c5a716d9180 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.821668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc37e9-81a1-4200-ace1-0760117b54f6 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.826929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.827522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.827800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.828067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.828727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.830332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.840444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 792/1582] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:22.841522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.851438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764acd60-e363-4762-8c14-997e4cb0a004 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.856229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.856809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.857091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.857370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.857997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.859906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.860525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.861307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.861680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.862045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.862792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.864584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:22.876265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 791/1583] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:22.877229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:22.887795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72463f3-7689-4a87-90af-ad44a25c37d4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:22.893078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:22.893657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:22.893939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:22.894225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:22.894944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:22.896858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.015012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 793/1584] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/users => generated 278 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:23.015012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.022234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f637c3f-a8d1-4ace-ac9b-8005c2d28812 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.026946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.027555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.027833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.028105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.028801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.030398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.041495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 794/1585] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.042405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.052781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 792/1586] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:22 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:23.053401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765692fc-44c6-4c0c-91f2-16e4cee2a0ed admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.054033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.060817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.061500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.077890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.077890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.077890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.078205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5f21e-2d8e-4f8c-a821-48fe003bac93 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.078292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.078376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.078376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.078376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.078376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.078687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.078687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.078913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:23.079000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:23.186521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 795/1587] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:23 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:23.186521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.186782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 793/1588] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:23 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:23.186782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.197938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be4af-b196-4494-81ca-068380d6cc34 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.249282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55993663-e819-495a-9ea4-f8d321801664 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.253988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.254323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.254600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.254878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.254984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.255152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.255262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.255488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.255861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.256132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.257487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.257776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.267914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 796/1589] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.268487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 794/1590] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.271794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.271938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.281655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387ccc-ee01-41b0-9878-6214dbf69582 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.282667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28090a-71b9-4452-8103-80a3954fbe05 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.286511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.287172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.287451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.287552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.287721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] PATH_INFO: `/v3/projects/2d5b2c4bde0d454c85b5eb0f3da6dffd/users/4f59476c8bdc4881b5f1a8b27071aebc/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.288102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.288379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.288695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.289660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.291341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.305877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 795/1591] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.307001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.309555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5b2c4bde0d454c85b5eb0f3da6dffd, user_id=4f59476c8bdc4881b5f1a8b27071aebc, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.312922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.318003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71dcf3-a4e3-4ffc-a94b-7ca29c48d8f5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.322769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.323384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.323665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.323935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.324613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.326254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.337013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 796/1592] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.337792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.347257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d795e-d17f-427f-880c-d1178acdd4ce admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.351897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.352455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.352757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.353027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.353647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.355217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.365685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 797/1593] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.366511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.376556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa086a-10aa-4a10-9c27-2791c4fc733c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.385579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 797/1594] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:23 2022] PUT /identity/v3/projects/2d5b2c4bde0d454c85b5eb0f3da6dffd/users/4f59476c8bdc4881b5f1a8b27071aebc/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:23.391884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.438860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.439324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.439605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.439879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.440532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.442253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.452069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 798/1595] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.452495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2335d66-4347-4444-b652-04d55f59b23d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.452905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.458349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.460790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.460790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.460790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.460790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.462189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.463437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e5dc-a113-4a86-8841-76f9c2f9597e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.468880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.469529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.469806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.470078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] PATH_INFO: `/v3/projects/27c8e372420a4076b855d46a4f8ff521/users/d318793a5fee43bdb1a043b30396884b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.487005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27c8e372420a4076b855d46a4f8ff521, user_id=d318793a5fee43bdb1a043b30396884b, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.489734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.566939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 798/1596] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:23 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:23.567773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.577970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 799/1597] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:23 2022] PUT /identity/v3/projects/27c8e372420a4076b855d46a4f8ff521/users/d318793a5fee43bdb1a043b30396884b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:23.578311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540eb651-c977-4c28-b267-e53c1920485d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.578859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.599952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:23.626306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:23.635095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.635704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.636012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.636283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.636955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.639224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.641699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd50cb-a8cf-4d2b-99a7-5c6a8e413062 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.647229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.647836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.648119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.648392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.649088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.649576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 799/1598] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.650424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.654227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.659868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bbc4ab-c7aa-49cd-94ad-cd79ba2922a8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.664810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.664978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 800/1599] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.665647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.665991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.666201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.666375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] PATH_INFO: `/v3/projects/5b00b496a30043b093f29a81fc0694c3/users/8b63da4b987e41618280b19220ab6e04/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.675912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f95e1-95df-40a2-9113-5ac24b343111 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.680953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.684443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.684443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.684443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.684443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.685414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b00b496a30043b093f29a81fc0694c3, user_id=8b63da4b987e41618280b19220ab6e04, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.685519 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2765a9-75d1-47e6-8831-4361235e78c7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.686837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.695291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 801/1600] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.696696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.696986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.697276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.739238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] MFA Rules not processed for user `4f59476c8bdc4881b5f1a8b27071aebc`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:23.740448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:23.806651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 800/1601] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:23 2022] PUT /identity/v3/projects/5b00b496a30043b093f29a81fc0694c3/users/8b63da4b987e41618280b19220ab6e04/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:23.816977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.831762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 802/1602] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:23.831762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.840811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d5033-db3c-41a2-9167-2ec01066ced3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.883925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba546-69c3-4741-8bc5-20607081c19e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.888843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.889454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.889722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.889986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.890619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.892210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97c9d3-b10c-4e3e-b393-05b5ede41297 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.902586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 801/1603] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.903889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.904246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.904592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.907676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.908364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.908735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.909009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.909656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.911225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.923287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 803/1604] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:23.924185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:23.924303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.925431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:23.933539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 802/1605] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:23.934314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:23.936865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31beda-6a98-49f7-82ac-918ca2588f66 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.942250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.942984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.943329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.943667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] PATH_INFO: `/v3/projects/e52495240cf04548a21b411c32bbe1e1/users/51729e82a3784d04aef89ed83f1576bd/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.945155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d733959-caa3-4fe0-b1ce-e65a923b46a4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:23.950550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:23.951131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:23.951406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:23.951678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] PATH_INFO: `/v3/projects/a0132d93372a452d8b399dd12d16c2e1/users/28d670dfd1994727ad9eafa33e7019ee/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:23.959961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52495240cf04548a21b411c32bbe1e1, user_id=51729e82a3784d04aef89ed83f1576bd, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.962740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:23.966220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0132d93372a452d8b399dd12d16c2e1, user_id=28d670dfd1994727ad9eafa33e7019ee, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:23.969852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44f307-8a1d-4222-9b5a-3993363db80d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.033362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 804/1606] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:23 2022] PUT /identity/v3/projects/e52495240cf04548a21b411c32bbe1e1/users/51729e82a3784d04aef89ed83f1576bd/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:24.034240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.043132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:24.063774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 803/1607] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:23 2022] PUT /identity/v3/projects/a0132d93372a452d8b399dd12d16c2e1/users/28d670dfd1994727ad9eafa33e7019ee/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:24.065055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.065340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.065615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.092971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] MFA Rules not processed for user `8b63da4b987e41618280b19220ab6e04`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.094249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.097459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b35cb-e027-40e4-b8af-9fa3557fe819 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.161487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.162442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.162723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.162994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] PATH_INFO: `/v3/projects/728d1256cb6141a197dcfc82d7847e67/users/7131191c2bf642bb9c698d6d740d5cff/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.165781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 804/1608] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.170754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.180323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] RBAC: Authorizing `identity:create_grant(project_id=728d1256cb6141a197dcfc82d7847e67, user_id=7131191c2bf642bb9c698d6d740d5cff, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.183121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.183919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b0ddd-7a8a-48b3-8b39-43bc3081f2bc None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.189686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.190394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.190683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.190963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.191599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.193550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.267505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 805/1609] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:24 2022] PUT /identity/v3/projects/728d1256cb6141a197dcfc82d7847e67/users/7131191c2bf642bb9c698d6d740d5cff/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:24.268375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.298343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:24.306900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 805/1610] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:24.306900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.332168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa2bf0-b206-422f-af9e-a0a1c084973e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.367309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7a26-587c-40ed-9038-6cc8d1c73594 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.372013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.372699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.372985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.373260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] PATH_INFO: `/v3/projects/ff7f41efa6f74da28389e2cac8347db8/users/eef19e9f8dc24a7492206471c79be73a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.374311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:24.388026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7f41efa6f74da28389e2cac8347db8, user_id=eef19e9f8dc24a7492206471c79be73a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.390732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.393175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.393848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.394127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.394402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] PATH_INFO: `/v3/projects/601816703e524795ab40b1dea9393870/users/c772000142f14498933a685d3855bc02/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.411210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=601816703e524795ab40b1dea9393870, user_id=c772000142f14498933a685d3855bc02, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.413984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226235fe-9b66-4989-a608-d967d0f7abfc admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.481345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 806/1611] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:24 2022] PUT /identity/v3/projects/ff7f41efa6f74da28389e2cac8347db8/users/eef19e9f8dc24a7492206471c79be73a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:24.483048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.510374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 806/1612] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:24 2022] PUT /identity/v3/projects/601816703e524795ab40b1dea9393870/users/c772000142f14498933a685d3855bc02/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:24.512231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ffe1c-0c22-482b-ab60-f1afd13cd6ea None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.513270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ffe1c-0c22-482b-ab60-f1afd13cd6ea None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.513744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ffe1c-0c22-482b-ab60-f1afd13cd6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.534221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-764ffe1c-0c22-482b-ab60-f1afd13cd6ea None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.535389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-764ffe1c-0c22-482b-ab60-f1afd13cd6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.550662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58f39d-5c69-484c-a5f0-2bfbb3a89768 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.596758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 807/1613] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.598130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.598364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.598498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.598709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.599020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.599297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.599568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.609426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.611279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5940f-fc0e-4057-ad19-d2e7d6ccacc1 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.621575 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 807/1614] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:24.622970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569e45-c232-4c63-a2dd-0b8c7853850b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.623364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569e45-c232-4c63-a2dd-0b8c7853850b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.623814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569e45-c232-4c63-a2dd-0b8c7853850b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.627255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] MFA Rules not processed for user `28d670dfd1994727ad9eafa33e7019ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.628468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.628830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 808/1615] 10.222.0.72 () {54 vars in 962 bytes} [Wed Aug 24 07:55:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:24.630463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.630911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.631190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.660386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] MFA Rules not processed for user `7131191c2bf642bb9c698d6d740d5cff`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.661642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.695137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 808/1616] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.696028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.706931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ebeaa-d232-4183-a2d1-212d35ce4898 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.709533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:24.711724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.712323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.712603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.712908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.713663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.715499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.734727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 809/1617] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.735686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.746112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94949db8-e8d8-486b-bd4d-22f76a1aa2e5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.755859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.756419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.756419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcafe20-7e64-4023-b941-96f2d70263a3 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.767425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 810/1618] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:24.769050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.769319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.769588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.793188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.793188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.801983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 811/1619] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.805506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.805506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.820069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc8121-c624-4837-9957-a79c373053ef None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.824352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.825759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.826181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.827651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.840926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.842913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7b829-94ed-47de-8025-32fb2a61668a service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.852622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 812/1620] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:24.853873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfdd6b-c5c5-4911-834c-556d8305d5c2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.854160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfdd6b-c5c5-4911-834c-556d8305d5c2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.854654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfdd6b-c5c5-4911-834c-556d8305d5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.868068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 809/1621] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:24.868068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.878640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39893b9-c976-4cf0-8334-15ed8d7e9fd6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.883270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.884094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.884609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.885012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.885711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.887779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:24.894741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-73bfdd6b-c5c5-4911-834c-556d8305d5c2 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:24.895955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-73bfdd6b-c5c5-4911-834c-556d8305d5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:24.963189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 813/1622] 10.222.0.72 () {58 vars in 1059 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:24.964448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521142e4-de9b-4d84-838f-3d35e45f7040 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.964754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521142e4-de9b-4d84-838f-3d35e45f7040 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.965025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521142e4-de9b-4d84-838f-3d35e45f7040 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.966573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 814/1623] 10.222.0.72 () {54 vars in 962 bytes} [Wed Aug 24 07:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:24.967246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521142e4-de9b-4d84-838f-3d35e45f7040 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:24.980096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521142e4-de9b-4d84-838f-3d35e45f7040 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:24.985492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:24.986189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:24.986470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:24.986742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] PATH_INFO: `/v3/projects/ff7f41efa6f74da28389e2cac8347db8/users/eef19e9f8dc24a7492206471c79be73a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:24.988932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7f41efa6f74da28389e2cac8347db8, user_id=eef19e9f8dc24a7492206471c79be73a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:24.991590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.004246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 815/1624] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:24 2022] PUT /identity/v3/projects/ff7f41efa6f74da28389e2cac8347db8/users/eef19e9f8dc24a7492206471c79be73a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:25.008368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.013493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 810/1625] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:24 2022] POST /identity/v3/projects => generated 355 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:25.016430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.049450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:25.096748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06a89c-5322-440c-82aa-14120c344ed0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.110570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d407-3a1b-44d4-a595-9f6dbd62649f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.119946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.120955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.121760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.122782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.123783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.125964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436a552-2e58-4d2d-a559-6fc01f844dc2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.139534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 811/1626] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:25.141386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.142164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.142681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.164623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.165351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.165639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.165890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.168053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:25.169581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:25.177510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.179157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.192086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 816/1627] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:25.193146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.246787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 812/1628] 10.222.0.72 () {58 vars in 1059 bytes} [Wed Aug 24 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:25.255377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.265985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95a20f-e412-45e0-a89a-a3bb540ebbc3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.271241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.271763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da01ba-1936-4fe7-b493-bf5ab3c1578b service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.271893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.273411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.273411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.273411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.275159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.277454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.278077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.278404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.278749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.324464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:25.348601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.350411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec48db-3e6c-4725-8318-a3b490262fee service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.361457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 817/1629] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:25.363011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.363260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.363472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.393666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 813/1630] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:25 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:25.393666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.429008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26ad03-842d-4fd2-9956-1ef4759bd5b1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.439753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] MFA Rules not processed for user `eef19e9f8dc24a7492206471c79be73a`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:25.443369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.443718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:25.444570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.446886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.448044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] PATH_INFO: `/v3/projects/eea0f6e1b06747ac99e40715ccc8b259/users/59f2e9863440452ba76a604769e2ffc3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.480974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea0f6e1b06747ac99e40715ccc8b259, user_id=59f2e9863440452ba76a604769e2ffc3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.484434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.548168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 818/1631] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:25.549122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.562668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e53f65-83ab-4003-97e6-6f2ddb6e75a2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.568579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.569316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.569667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.570065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.618530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 814/1632] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:25 2022] PUT /identity/v3/projects/eea0f6e1b06747ac99e40715ccc8b259/users/59f2e9863440452ba76a604769e2ffc3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:25.618530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.640208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.643075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.667701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 819/1633] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:25.671853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.741857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587d3d6-3c98-49c9-9f27-0213025a5ae0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.752436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.754618 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.755233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.757634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.758514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.763069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.778098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:25.791056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 815/1634] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:25.791056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.802308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:25.812194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9aeb-a2ce-41d8-be50-32c0386a474f service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.814745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64644721-94ff-43ea-a295-7ffdd02d4c6d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.819988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.820752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.821104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.821716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.822761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.824729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63fe23-9657-4022-ad93-ece15a269a5a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.844321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 816/1635] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:25.847466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e022-c8be-4dec-8d79-14c276abf313 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.847838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e022-c8be-4dec-8d79-14c276abf313 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.849112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e022-c8be-4dec-8d79-14c276abf313 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.852746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 817/1636] 10.222.0.72 () {54 vars in 912 bytes} [Wed Aug 24 07:55:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:25.854868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e022-c8be-4dec-8d79-14c276abf313 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:25.881486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e022-c8be-4dec-8d79-14c276abf313 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:25.886610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.887335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.887642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.887925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] PATH_INFO: `/v3/projects/30660254fd4c478f9013dcebba9b3d75/users/fa46a12ba79f446da1936ec2e901d1c3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.906748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30660254fd4c478f9013dcebba9b3d75, user_id=fa46a12ba79f446da1936ec2e901d1c3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.909946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:25.910568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:25.910968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:25.911112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:25.925302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510a38-7258-4f2b-bfce-a9a7e8d02c18 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:25.986059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:25.987774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.004011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 820/1637] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:26.004950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.030029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102838c5-624c-4ec6-a4ee-25f80900227d service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.036339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.036812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.037092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.037361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] PATH_INFO: `/v3/projects/eea0f6e1b06747ac99e40715ccc8b259/users/59f2e9863440452ba76a604769e2ffc3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.040076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea0f6e1b06747ac99e40715ccc8b259, user_id=59f2e9863440452ba76a604769e2ffc3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.048684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.056835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 818/1638] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:25 2022] PUT /identity/v3/projects/30660254fd4c478f9013dcebba9b3d75/users/fa46a12ba79f446da1936ec2e901d1c3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:26.056835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087a77-2266-49f6-9e7f-a656b5a08d3b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.056835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087a77-2266-49f6-9e7f-a656b5a08d3b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.056835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc087a77-2266-49f6-9e7f-a656b5a08d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.062736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 821/1639] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:26 2022] PUT /identity/v3/projects/eea0f6e1b06747ac99e40715ccc8b259/users/59f2e9863440452ba76a604769e2ffc3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:26.063746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.100963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-bc087a77-2266-49f6-9e7f-a656b5a08d3b None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:26.100963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bc087a77-2266-49f6-9e7f-a656b5a08d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:26.173786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6d56c-01ff-4033-a920-a7618d974023 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.178844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.179466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.179744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.180012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.180795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.182294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340be577-ec37-4b29-99fd-5ed2a83f38e2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.198484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 822/1640] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:26.198484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.198484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.198910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 819/1641] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577905f6-d739-489c-ae59-0bae651e9ad8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577905f6-d739-489c-ae59-0bae651e9ad8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577905f6-d739-489c-ae59-0bae651e9ad8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 820/1642] 10.222.0.72 () {54 vars in 912 bytes} [Wed Aug 24 07:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:26.227322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577905f6-d739-489c-ae59-0bae651e9ad8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.242431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577905f6-d739-489c-ae59-0bae651e9ad8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.256612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.257512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.257909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.258449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] PATH_INFO: `/v3/projects/30660254fd4c478f9013dcebba9b3d75/users/fa46a12ba79f446da1936ec2e901d1c3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.260401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30660254fd4c478f9013dcebba9b3d75, user_id=fa46a12ba79f446da1936ec2e901d1c3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.263831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277801-c887-4974-b06f-b2a1c91c0d7f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.268982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] MFA Rules not processed for user `59f2e9863440452ba76a604769e2ffc3`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:26.270198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:26.274809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 821/1643] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:26 2022] PUT /identity/v3/projects/30660254fd4c478f9013dcebba9b3d75/users/fa46a12ba79f446da1936ec2e901d1c3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:26.276215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cf0a-4ad5-4e95-a97b-3e1f3cf688a8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.276557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cf0a-4ad5-4e95-a97b-3e1f3cf688a8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.285105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8cf0a-4ad5-4e95-a97b-3e1f3cf688a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.297169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-daf8cf0a-4ad5-4e95-a97b-3e1f3cf688a8 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:26.298370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-daf8cf0a-4ad5-4e95-a97b-3e1f3cf688a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:26.320502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:26.353121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 823/1644] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:26.362322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.389062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 822/1645] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:26.390548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.390988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.391148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.423150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] MFA Rules not processed for user `fa46a12ba79f446da1936ec2e901d1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:26.424430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:26.439329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956beab-aee6-4e5f-803d-d57f8f57889e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.443003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.443615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.443895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.444166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.455138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.456671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.469277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 824/1646] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:26.470367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.486902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9f424-b107-4583-9081-86f57c724e12 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.492764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.492764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.493053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.493315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.521113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.523398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.526188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 823/1647] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:26.540403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 825/1648] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:26.584139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:26.681672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:26.883626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:26.893512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e949d5-92c6-450a-88fa-2e3b7fdfa862 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:26.897360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:26.898033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:26.898310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:26.898596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:26.915812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:26.915812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:26.927030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 824/1649] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:27.003823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:27.013205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7a6cc-477e-4d9a-a2f1-85f69acfdc9a service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:27.021056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:27.021056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:27.021056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:27.021056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:27.404337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:27.404337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:27.404337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 826/1650] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:28.013447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:28.015731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:28.026776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa63673-483c-46ca-87af-683cb3d1cfba service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:28.038839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:28.040282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:28.040761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:28.041224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:28.176068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ecf51-44e5-4f3d-840a-ef9547c3b372 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:28.176068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:28.176068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:28.176068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] PATH_INFO: `/v3/users/7131191c2bf642bb9c698d6d740d5cff` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7131191c2bf642bb9c698d6d740d5cff)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: WARNING py.warnings [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: warnings.warn(msg) Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: Aug 24 07:55:28.176793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:28.177524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:28.177524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a1e16-f98c-4c24-9330-5e9f9c7ad27a service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:28.188740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 825/1651] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:28.452935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7131191c2bf642bb9c698d6d740d5cff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:28.456257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7131191c2bf642bb9c698d6d740d5cff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:28.479591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7131191c2bf642bb9c698d6d740d5cff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:28.571880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7131191c2bf642bb9c698d6d740d5cff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:28.604093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7131191c2bf642bb9c698d6d740d5cff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:28.611791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 827/1652] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:28 2022] DELETE /identity/v3/users/7131191c2bf642bb9c698d6d740d5cff => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:28.616479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:28.617006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:28.617465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:28.647825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:28.652814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:28.825322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 826/1653] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:28.910026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:28.912860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:29.137357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c4612-6d8f-4c4a-b80a-0db6dfc71857 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:29.143001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:29.143977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:29.144437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:29.144889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:29.159074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:29.167442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:29.169853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:29.182790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 828/1654] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:30.125033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:30.126785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.143727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3498f75-cf88-4a37-aa75-b3142c3c57a4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:30.158678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:30.160453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.178150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:30.180954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c76d24-d0a8-46d5-8eb6-3b72cd9a71e7 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:30.190391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.190391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.190391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] PATH_INFO: `/v3/projects/728d1256cb6141a197dcfc82d7847e67` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.190391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728d1256cb6141a197dcfc82d7847e67)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:30.191820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: WARNING py.warnings [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 07:55:30.191820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: warnings.warn(msg) Aug 24 07:55:30.191820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: Aug 24 07:55:30.194753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:30.207504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:30.209050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.209469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.210055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] PATH_INFO: `/v3/users/4f59476c8bdc4881b5f1a8b27071aebc` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.212019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f59476c8bdc4881b5f1a8b27071aebc)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:30.214181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:30.480868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:30.497690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f59476c8bdc4881b5f1a8b27071aebc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.498793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f59476c8bdc4881b5f1a8b27071aebc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.527940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f59476c8bdc4881b5f1a8b27071aebc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.548045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728d1256cb6141a197dcfc82d7847e67', 'request_id': 'req-3a6049b2-3f92-4572-9a40-a64b21422806'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.567085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728d1256cb6141a197dcfc82d7847e67', 'request_id': 'req-3a6049b2-3f92-4572-9a40-a64b21422806'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.625164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728d1256cb6141a197dcfc82d7847e67', 'request_id': 'req-3a6049b2-3f92-4572-9a40-a64b21422806'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.650884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f59476c8bdc4881b5f1a8b27071aebc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.681104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f85f59a3-faff-4df0-84e7-d4d417f37bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f59476c8bdc4881b5f1a8b27071aebc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.701536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 829/1655] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:30 2022] DELETE /identity/v3/users/4f59476c8bdc4881b5f1a8b27071aebc => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:30.703102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60f8f-7233-4af2-b65a-2056212c396e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.703394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60f8f-7233-4af2-b65a-2056212c396e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.703670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d60f8f-7233-4af2-b65a-2056212c396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.764525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] The token cache is being invalidate because project 728d1256cb6141a197dcfc82d7847e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:30.765159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.766365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3a6049b2-3f92-4572-9a40-a64b21422806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:30.768459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 827/1656] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:30 2022] DELETE /identity/v3/projects/728d1256cb6141a197dcfc82d7847e67 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:30.770744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-34d60f8f-7233-4af2-b65a-2056212c396e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:30.773257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-34d60f8f-7233-4af2-b65a-2056212c396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:30.774837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:30.775291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:30.783420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:30.857566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:30.858955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.036047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 830/1657] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:31.063577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 828/1658] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:31.065589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.065974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.066349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.133664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:31.136246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.175006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 831/1659] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:31.178961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.199657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bc2c6-9efa-4dff-9ff4-ac989006d553 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.240857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.246369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.250723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.254713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.254713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:31.254713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.254713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.302038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 829/1660] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:31.304877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.326636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d212741-4189-4f41-a578-44f7b84ad4d7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.362396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.363305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.363729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.364131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.365387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.372844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.506259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.508804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:31.549531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 832/1661] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:31.553791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.660275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:31.662180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31aee5-d151-49ed-b970-4b0ba30b0450 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.697089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61eeb-9be3-42a8-ad61-8d1245ccb9a6 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.704416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.705358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.706379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.706832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.708879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.711980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.799501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.800514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.800863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.801202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.822407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.823938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.849983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 830/1662] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:55:31.892539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 833/1663] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:31 2022] POST /identity/v3/users => generated 291 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:31.897574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.917182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f356f0-1dcf-43da-bd15-86971b20f9bb service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.929907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.931031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.931314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.931591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.932331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:31.937216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:31.953601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 831/1664] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:31.958573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:31.978953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1afc05-48b5-4590-a8ff-71063e599380 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:31.989441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:31.990328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:31.991855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:31.997423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:31.998060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.000496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341e944c-efb5-4c5a-881d-1bfa51248518 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.014419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 834/1665] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:32.018646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.029206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:32.039875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2f6a-5818-4235-92b7-9d9712e0b2a7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.073844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.074527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.075344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.075625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] PATH_INFO: `/v3/projects/d2a4b5afd95d436f916d9cc485d77b79/users/fd45688f32194dc59fdef259b1cf81f3/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.120605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a4b5afd95d436f916d9cc485d77b79, user_id=fd45688f32194dc59fdef259b1cf81f3, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.123380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.250732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 832/1666] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:32 2022] PUT /identity/v3/projects/d2a4b5afd95d436f916d9cc485d77b79/users/fd45688f32194dc59fdef259b1cf81f3/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:32.261111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.261641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.261877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.317106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] MFA Rules not processed for user `fd45688f32194dc59fdef259b1cf81f3`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:32.317106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:32.451545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 835/1667] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:32.453700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.613910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98592520-e696-461e-ae11-16f175874f3d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:32.628311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:32.629652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:32.630103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:32.630527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:32.631555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:32.636015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:32.833924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 833/1668] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:32 2022] POST /identity/v3/projects => generated 381 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:32.837702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.867008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:32.867008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:32.869465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a72be-7039-4c70-b3a3-7e4202f65be6 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.009765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b3123-8333-42e5-9930-81b44347f4b5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.025580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.033239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.033424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.033841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.033841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] PATH_INFO: `/v3/projects/2d5b2c4bde0d454c85b5eb0f3da6dffd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.035507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.035507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.035507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.035507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.038826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.043998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5b2c4bde0d454c85b5eb0f3da6dffd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.046005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.223957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5b2c4bde0d454c85b5eb0f3da6dffd', 'request_id': 'req-789951ef-5a06-4650-86db-b5cad367765f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:33.243257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 836/1669] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:32 2022] POST /identity/v3/users => generated 297 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:55:33.243257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:33.255112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5b2c4bde0d454c85b5eb0f3da6dffd', 'request_id': 'req-789951ef-5a06-4650-86db-b5cad367765f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:33.301975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5b2c4bde0d454c85b5eb0f3da6dffd', 'request_id': 'req-789951ef-5a06-4650-86db-b5cad367765f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:33.327262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9a778-0f32-47a1-a57a-32cd5ed956ea admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.471516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] The token cache is being invalidate because project 2d5b2c4bde0d454c85b5eb0f3da6dffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:33.472014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:33.472624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:33.483227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 834/1670] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:32 2022] DELETE /identity/v3/projects/2d5b2c4bde0d454c85b5eb0f3da6dffd => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:33.484442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:33.486124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:33.599312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.600008 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.600291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.604136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.607133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.614251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.656905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 837/1671] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:33.663166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:33.668230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:33.719324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492587-9fa1-45a5-bc6a-538fa3ee1438 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.726067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.726968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.727647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.728074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] PATH_INFO: `/v3/projects/46f5b7ef37a142e6879ca69dba9a9560/users/62ef39d3e90d46afb6bfdea7a50437c7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.785635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789951ef-5a06-4650-86db-b5cad367765f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:33.803797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:33.804626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:33.805089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:33.805462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] PATH_INFO: `/v3/users/59f2e9863440452ba76a604769e2ffc3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:33.811855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f2e9863440452ba76a604769e2ffc3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.814201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.862926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f5b7ef37a142e6879ca69dba9a9560, user_id=62ef39d3e90d46afb6bfdea7a50437c7, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:33.877405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:33.899383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:34.052174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 838/1672] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:33 2022] PUT /identity/v3/projects/46f5b7ef37a142e6879ca69dba9a9560/users/62ef39d3e90d46afb6bfdea7a50437c7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:34.053910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:34.057752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:34.096196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f2e9863440452ba76a604769e2ffc3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.113335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f2e9863440452ba76a604769e2ffc3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.116413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f2e9863440452ba76a604769e2ffc3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.120117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f2e9863440452ba76a604769e2ffc3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.208667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0691fe1-3e00-44bf-9663-0298377a1343 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:34.223293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:34.223967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:34.228224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:34.228987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] PATH_INFO: `/v3/users/0d774440fd2141f2b9407520e0cbc94f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:34.230706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d774440fd2141f2b9407520e0cbc94f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:34.237695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:34.275470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f2e9863440452ba76a604769e2ffc3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.314612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 835/1673] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:33 2022] DELETE /identity/v3/users/59f2e9863440452ba76a604769e2ffc3 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:34.319740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:34.322142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:34.428399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4137df7-5249-4bc1-8365-8010ea9dab8c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:34.440978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:34.441857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:34.442295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:34.444202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:34.444202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:34.446039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:34.459716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 836/1674] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:34.464112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:34.476116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1678e-08dd-4220-bda6-5c4ecab30495 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:34.481595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:34.487256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:34.490146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:34.490146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:34.490146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] PATH_INFO: `/v3/projects/46f5b7ef37a142e6879ca69dba9a9560/users/62ef39d3e90d46afb6bfdea7a50437c7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:34.491858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f5b7ef37a142e6879ca69dba9a9560, user_id=62ef39d3e90d46afb6bfdea7a50437c7, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:34.498176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18262823-8e17-4720-8a9b-4b326fb1f0e1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:34.518473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 837/1675] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:34 2022] PUT /identity/v3/projects/46f5b7ef37a142e6879ca69dba9a9560/users/62ef39d3e90d46afb6bfdea7a50437c7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:34.530074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:34.530825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:34.531393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:34.595398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] MFA Rules not processed for user `62ef39d3e90d46afb6bfdea7a50437c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:34.598692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:34.682254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d774440fd2141f2b9407520e0cbc94f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.684860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d774440fd2141f2b9407520e0cbc94f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.711477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d774440fd2141f2b9407520e0cbc94f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.773840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 838/1676] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:55:34.790109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:34.794455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:34.811298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d774440fd2141f2b9407520e0cbc94f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.830244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-74055009-9dda-430f-8657-a1c4a80d85fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d774440fd2141f2b9407520e0cbc94f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:34.849691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 839/1677] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:34 2022] DELETE /identity/v3/users/0d774440fd2141f2b9407520e0cbc94f => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:34.854460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74cd2b-0eb6-4bc3-b931-e6a6399cbbe1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:34.854929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74cd2b-0eb6-4bc3-b931-e6a6399cbbe1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:34.855396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74cd2b-0eb6-4bc3-b931-e6a6399cbbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:34.905903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2f74cd2b-0eb6-4bc3-b931-e6a6399cbbe1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:34.905903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2f74cd2b-0eb6-4bc3-b931-e6a6399cbbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:34.959404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3725196-7959-45e1-ab04-a6b5ebddbd90 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:35.033203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 840/1678] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:55:35.033203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.033203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.048880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.097434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] MFA Rules not processed for user `d318793a5fee43bdb1a043b30396884b`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:35.098958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:35.113031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:35.118537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.123498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.123498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.205564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 841/1679] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:35.221855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:35.232856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:35.296910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:35.298456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:35.325013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 839/1680] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:35.327108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:35.327795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788efaa-2a98-4937-aa81-a33c175d042e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:35.332153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:35.332743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.333251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.334119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:35.339279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.350743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:35.352406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:35.369101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 842/1681] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:35.554969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf24197-01dc-4263-be9f-9f471d66885c service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:35.577109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:35.577109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.577109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.577109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.615124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:35.617114 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:35.636714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 840/1682] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:35.684411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:35.714753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7ee81-13c7-44bb-9c10-d1ff3d77cc34 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:35.725101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:35.725724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:35.726007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:35.726389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:35.730262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:35.882442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:35.884788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:35.900335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 843/1683] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:36.142570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:36.339958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:36.344201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:36.382470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d605ae-f7bb-4ce7-91d4-95c5c695c90c service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:36.384437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:36.388925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:36.395691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:36.399275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:36.406502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:36.407599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] PATH_INFO: `/v3/projects/eea0f6e1b06747ac99e40715ccc8b259` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:36.410291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea0f6e1b06747ac99e40715ccc8b259)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:36.414695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:36.419319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f48162-9c5c-48d3-9326-fa4723b581b4 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:36.425398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:36.426364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:36.426791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:36.427266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] PATH_INFO: `/v3/projects/7f6d7dd049f440828cec4efd5d4ef101` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:36.428682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f6d7dd049f440828cec4efd5d4ef101)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:36.430962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:36.554797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea0f6e1b06747ac99e40715ccc8b259', 'request_id': 'req-2f391998-a37c-4489-a1b5-1e4365eba54f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.573911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea0f6e1b06747ac99e40715ccc8b259', 'request_id': 'req-2f391998-a37c-4489-a1b5-1e4365eba54f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.584888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f6d7dd049f440828cec4efd5d4ef101', 'request_id': 'req-2c795903-0c7e-4b2b-af1a-af29f71b06ec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.596124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea0f6e1b06747ac99e40715ccc8b259', 'request_id': 'req-2f391998-a37c-4489-a1b5-1e4365eba54f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.732503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f6d7dd049f440828cec4efd5d4ef101', 'request_id': 'req-2c795903-0c7e-4b2b-af1a-af29f71b06ec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.739925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f6d7dd049f440828cec4efd5d4ef101', 'request_id': 'req-2c795903-0c7e-4b2b-af1a-af29f71b06ec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.763857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] The token cache is being invalidate because project 7f6d7dd049f440828cec4efd5d4ef101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:36.764329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.765694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.773960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 844/1684] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:36 2022] DELETE /identity/v3/projects/7f6d7dd049f440828cec4efd5d4ef101 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:36.779202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:36.779202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:36.840792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] The token cache is being invalidate because project eea0f6e1b06747ac99e40715ccc8b259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:36.841314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.842530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2f391998-a37c-4489-a1b5-1e4365eba54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:36.848493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 841/1685] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:36 2022] DELETE /identity/v3/projects/eea0f6e1b06747ac99e40715ccc8b259 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:36.852849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa62cd-904f-4bd3-aad3-ee80a9e69468 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:36.853150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa62cd-904f-4bd3-aad3-ee80a9e69468 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:36.853422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa62cd-904f-4bd3-aad3-ee80a9e69468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:36.901392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795903-0c7e-4b2b-af1a-af29f71b06ec admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:36.903528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-85fa62cd-904f-4bd3-aad3-ee80a9e69468 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:36.905450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-85fa62cd-904f-4bd3-aad3-ee80a9e69468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:36.913282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:36.914312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:36.914587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:36.914855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] PATH_INFO: `/v3/users/eef19e9f8dc24a7492206471c79be73a` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:36.916721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef19e9f8dc24a7492206471c79be73a)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:36.919578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:36.934411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 842/1686] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:36.935720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:36.936135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:36.936568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:36.975948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:36.977837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:36.993546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 843/1687] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:36.995483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.010306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc89510-fe52-46e6-94df-105947ac32ff None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.018763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.020151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.020622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.021072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:37.021890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:37.022691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:37.025694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:37.049040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 844/1688] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:37.050275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.062447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bce17-65ef-4083-87d5-db3b27d74596 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.078884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.079595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.079876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.080330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:37.081015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:37.082943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:37.244731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 845/1689] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:37 2022] POST /identity/v3/projects => generated 389 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:37.252933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.278278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef19e9f8dc24a7492206471c79be73a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:37.280094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef19e9f8dc24a7492206471c79be73a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:37.302018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef19e9f8dc24a7492206471c79be73a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:37.352900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:37.411473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef19e9f8dc24a7492206471c79be73a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:37.442482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef19e9f8dc24a7492206471c79be73a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:37.453503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 845/1690] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:36 2022] DELETE /identity/v3/users/eef19e9f8dc24a7492206471c79be73a => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:37.486046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45e295-c566-474d-ab47-3df6a25ee6fd admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.615240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.618455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.620970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.624030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:37.626243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:37.630154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:37.794891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.801485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:37.804810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 846/1691] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:37 2022] POST /identity/v3/users => generated 295 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:37.835655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be738a8e-b724-4789-9465-d63c1d4c6c98 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.845583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.847147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.849863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.850185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:37.851029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:37.857615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:37.891358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.905822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e259d-9cd0-4f9b-b797-e03d386d8a4d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.906855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 846/1692] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:37.914408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.915005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.915994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.916453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:37.919561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:37.922571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98aa497-5bd2-46ab-9668-1fe69d31529d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:37.956216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:37.970103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 847/1693] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:37.971859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6fe24-f990-47da-9071-cb4268c17ec0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:37.985309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:37.986155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:37.986518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:37.986862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] PATH_INFO: `/v3/projects/e95df705878f4d9bbcec5d24e0125e58/users/fd3e795dcd5442e094863cb3200fea58/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:38.034655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95df705878f4d9bbcec5d24e0125e58, user_id=fd3e795dcd5442e094863cb3200fea58, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:38.037960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:38.194551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 847/1694] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:37 2022] PUT /identity/v3/projects/e95df705878f4d9bbcec5d24e0125e58/users/fd3e795dcd5442e094863cb3200fea58/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:38.211000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:38.211922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:38.212403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:38.361013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] MFA Rules not processed for user `fd3e795dcd5442e094863cb3200fea58`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:38.362974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:38.593276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:38.595402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:38.663169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 848/1695] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:38.667973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:38.766232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bf259-c0ee-4943-b0ba-92eea9c45ccb admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:38.779933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:38.780981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:38.781957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:38.782654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] PATH_INFO: `/v3/users/8b63da4b987e41618280b19220ab6e04` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:38.784338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b63da4b987e41618280b19220ab6e04)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:38.787146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:38.865339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9699c8-cc3c-4ab4-9ed7-ec8c1b73f093 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:38.869266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:38.880149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:38.880889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:38.881324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:38.885939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:38.886636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:38.888607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:39.061280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 849/1696] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:38 2022] POST /identity/v3/projects => generated 391 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:39.062390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:39.068998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.093516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b63da4b987e41618280b19220ab6e04'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.104436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b63da4b987e41618280b19220ab6e04'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.128236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b63da4b987e41618280b19220ab6e04'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.250556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b63da4b987e41618280b19220ab6e04'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.269907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16195908-951c-4d07-bea8-c19cc2b5dcd6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:39.273305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b63da4b987e41618280b19220ab6e04'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.281455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:39.284410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.284536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 848/1697] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:38 2022] DELETE /identity/v3/users/8b63da4b987e41618280b19220ab6e04 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:39.285608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:39.286740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.286740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] PATH_INFO: `/v3/projects/ff7f41efa6f74da28389e2cac8347db8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.290190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.290340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7f41efa6f74da28389e2cac8347db8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:39.295040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:39.446132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7f41efa6f74da28389e2cac8347db8', 'request_id': 'req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.455617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7f41efa6f74da28389e2cac8347db8', 'request_id': 'req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.457232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3ff6-a474-41a2-9b25-546a979fbbc0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:39.468005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7f41efa6f74da28389e2cac8347db8', 'request_id': 'req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.560732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] The token cache is being invalidate because project ff7f41efa6f74da28389e2cac8347db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:39.561004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.562589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a32ba3-1c8f-4431-8006-e6474ac7bd82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:39.583031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 850/1698] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:39 2022] DELETE /identity/v3/projects/ff7f41efa6f74da28389e2cac8347db8 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:39.585440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.586223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.590181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.600496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:39.601213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.601500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.601951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.602763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:39.605282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9d8ec-aa8c-41e9-856e-8fb62e8e5abe admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:39.682080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] MFA Rules not processed for user `51729e82a3784d04aef89ed83f1576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:39.699336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.752864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 849/1699] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/users => generated 302 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:39.755110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48321a65-4b8b-45eb-a2bf-176c7b718600 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.755875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48321a65-4b8b-45eb-a2bf-176c7b718600 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.757294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48321a65-4b8b-45eb-a2bf-176c7b718600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.774133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:39.805207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-48321a65-4b8b-45eb-a2bf-176c7b718600 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:39.806449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-48321a65-4b8b-45eb-a2bf-176c7b718600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.827110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 850/1700] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:39.835813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9280-ee75-437a-9c88-602312e2ae0a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.842212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9280-ee75-437a-9c88-602312e2ae0a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.842212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9280-ee75-437a-9c88-602312e2ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.850035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 851/1701] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:39.851424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:39.853103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.883431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8de0c-bd15-49df-9807-9d977fca25c3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:39.890273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-349a9280-ee75-437a-9c88-602312e2ae0a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:39.892503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-349a9280-ee75-437a-9c88-602312e2ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.898798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:39.899713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.900089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.900604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.901382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:39.903763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefb496-1d05-4d78-aa58-c9647bc17954 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:39.914997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 852/1702] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:39.916937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.917427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 851/1703] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:39.917639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.918934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.919321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:39.919602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:39.919876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:39.948571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] MFA Rules not processed for user `51729e82a3784d04aef89ed83f1576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:39.950015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.952906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] MFA Rules not processed for user `c772000142f14498933a685d3855bc02`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:39.954364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.960244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 852/1704] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:39.962993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:39.970522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:39.971364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:40.076933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 853/1705] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:40.082977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7696e-c64a-4dc6-9c0a-b00c430cc49b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.091054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.097828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.097828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.097828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.098126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.113385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.114970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6074973-aa4a-4247-a324-0a17b779d0fa service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.124015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390ba2e-4029-43fd-8f12-8ee91972a4af None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.134130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.134886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.135929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.137550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 853/1706] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:40.137996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] PATH_INFO: `/v3/projects/6a217e5dacbf48dd8545bf4a68444c59/users/fd98dddf597d4355822346ac9bb59ae8/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.141597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.144221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.144525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.173040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a217e5dacbf48dd8545bf4a68444c59, user_id=fd98dddf597d4355822346ac9bb59ae8, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.179206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfe151-d595-4f84-8cbf-6583d399b73e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.189215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:40.191078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:40.201229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 854/1707] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:40.202347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.205327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:40.285384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 854/1708] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:40 2022] PUT /identity/v3/projects/6a217e5dacbf48dd8545bf4a68444c59/users/fd98dddf597d4355822346ac9bb59ae8/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:40.285384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.285384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.286623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.323791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] MFA Rules not processed for user `c772000142f14498933a685d3855bc02`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:40.323791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:40.395673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843af18-90bc-4057-a36b-1a94eae86999 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.446396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 855/1709] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:40.448013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.558188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d59d70-4459-498b-8350-5c7bf58072c7 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.558543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.559831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.561622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.562354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.568176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.569811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.571569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.572083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.573046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:40.575106 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.576976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.599105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 856/1710] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:40.600702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.624859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105fb7-0ec9-4c45-97ac-7dd12f7b1407 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.639642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.641084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.643135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:40.643762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.644242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.646115 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.650527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.675337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 857/1711] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:40.676429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.678156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.680004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.696245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:40.706145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 855/1712] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:40.707376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.743143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe776c6f-df92-480b-b14b-ccc09936df91 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.749880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd9e2c-a254-4b0d-a350-bfeea70a4803 service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.757531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.760050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.762658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.763117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.763277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.763964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.764387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.765379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.767601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.772539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.794009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.800193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.817994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 858/1713] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:40.821305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.844746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6be29-bee2-4d1d-b17f-024237ae32bb service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:40.870460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:40.871185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.871604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.871916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] PATH_INFO: `/v3/projects/6a217e5dacbf48dd8545bf4a68444c59/users/fd98dddf597d4355822346ac9bb59ae8/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.874039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a217e5dacbf48dd8545bf4a68444c59, user_id=fd98dddf597d4355822346ac9bb59ae8, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:40.877062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e327cef-e6cd-4aad-b353-258129a6944e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:40.913478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 859/1714] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:40 2022] PUT /identity/v3/projects/6a217e5dacbf48dd8545bf4a68444c59/users/fd98dddf597d4355822346ac9bb59ae8/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:40.925282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:40.925845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:40.926498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:40.934335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 856/1715] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:40.939961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:40.998235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] MFA Rules not processed for user `fd98dddf597d4355822346ac9bb59ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:40.999966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.082798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321adfd-8671-471e-8dc8-025d8c65326e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.099283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.099283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.099283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.099283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.099819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.102796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 860/1716] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:41.115000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.116771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.121019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.223115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 857/1717] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:40 2022] POST /identity/v3/users => generated 284 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:41.225521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.229783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.239567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3851f-f077-48ce-896b-af5dcce67bc5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.256121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bc5-f1ba-4bde-baa6-de29161be3d1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.260204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.260958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.261247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.261517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.279820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.280473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.280776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.281052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] PATH_INFO: `/v3/projects/5b00b496a30043b093f29a81fc0694c3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.283072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.284235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b00b496a30043b093f29a81fc0694c3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.285959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.292077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.329153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 861/1718] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:41.334680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.367911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32426373-bca8-4ea4-9dd9-76bc4f97f518 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.381485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.382633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.383089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.383643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.384671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.386424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39db29-f0db-4183-aac3-bc8a80ce7a61 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.401320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 862/1719] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:41.403058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.403470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.403900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.434192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] MFA Rules not processed for user `d318793a5fee43bdb1a043b30396884b`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:41.435922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.461349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b00b496a30043b093f29a81fc0694c3', 'request_id': 'req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:41.525508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b00b496a30043b093f29a81fc0694c3', 'request_id': 'req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:41.534249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b00b496a30043b093f29a81fc0694c3', 'request_id': 'req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:41.539036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 863/1720] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:41.540146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.572490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] The token cache is being invalidate because project 5b00b496a30043b093f29a81fc0694c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:41.572825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:41.575514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:41.581543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 858/1721] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:41 2022] DELETE /identity/v3/projects/5b00b496a30043b093f29a81fc0694c3 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:41.591213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.594416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.624678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ecdf5-0fd9-4980-b671-d1a8521ea1dc None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.630292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.630998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.631342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.632072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.632984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.634670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56868381-d040-42b9-ba43-e098bc084f2b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.662404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 864/1722] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:41.664340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.664794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.665204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.674302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:41.692199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:41.697929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.706578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a33be-5bf4-4025-a96d-f35eb7199cbd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.711077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.711861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.712284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.712753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.715391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.730490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 865/1723] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:41.730490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:41.730490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.750266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2ce4e-5458-4029-a3c8-6eaaaf2c6d3a None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:41.755718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:41.756562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.757082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.759639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] PATH_INFO: `/v3/projects/70557f70665b440ba434d06d57b04f78/users/550a229c588747438329b603b37b4764/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.785705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70557f70665b440ba434d06d57b04f78, user_id=550a229c588747438329b603b37b4764, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.793448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93e81f0-0b1e-42d4-96b3-a49475c3f7c1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.850037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:41.851646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca47b9-168f-4870-81b7-874fedf80ae2 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:41.862852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 859/1724] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:41.864149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8bf7c-5bb4-4395-b5e0-1f6a129217f1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.864440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8bf7c-5bb4-4395-b5e0-1f6a129217f1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.864864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8bf7c-5bb4-4395-b5e0-1f6a129217f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.877373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 866/1725] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:41 2022] PUT /identity/v3/projects/70557f70665b440ba434d06d57b04f78/users/550a229c588747438329b603b37b4764/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:41.878039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32013ef0-0bd2-4f88-980a-f3d670758916 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.878984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32013ef0-0bd2-4f88-980a-f3d670758916 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.878984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32013ef0-0bd2-4f88-980a-f3d670758916 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.880392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 867/1726] 10.222.0.72 () {54 vars in 912 bytes} [Wed Aug 24 07:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:41.882084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5b27-e4b8-4d02-98d6-c4842e693dad None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.882368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5b27-e4b8-4d02-98d6-c4842e693dad None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.882643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d5b27-e4b8-4d02-98d6-c4842e693dad None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.884109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 868/1727] 10.222.0.72 () {54 vars in 912 bytes} [Wed Aug 24 07:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:41.889666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a5a-97a7-40ef-a952-18ba2ed2ee8c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:41.890019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a5a-97a7-40ef-a952-18ba2ed2ee8c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:41.890906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa5a5a-97a7-40ef-a952-18ba2ed2ee8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:41.916236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a0b8bf7c-5bb4-4395-b5e0-1f6a129217f1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:41.917512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a0b8bf7c-5bb4-4395-b5e0-1f6a129217f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:41.937128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-79fa5a5a-97a7-40ef-a952-18ba2ed2ee8c None None] MFA Rules not processed for user `550a229c588747438329b603b37b4764`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:41.940947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-79fa5a5a-97a7-40ef-a952-18ba2ed2ee8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:42.013815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 860/1728] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:42.015844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.016348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.017286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.047225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 869/1729] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:42.050303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.050595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.051272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.098919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:42.100376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:42.114526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:42.117826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:42.224246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 861/1730] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:42.226103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.247052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 870/1731] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:42.248056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.266646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52302d-1c25-4137-9521-b403187854f6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.275913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.276762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.277040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.278046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.278525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c4552-1226-420a-870e-0a9fa9206f7a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.278846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.281647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.284245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.285369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.285947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.286553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.287402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:42.288028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.290714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.312702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 862/1732] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:42.314681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.317265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:42.332162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720008b2-5368-400b-90bc-833085bab16c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.341930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.342578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.342854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.343155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.362136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.363168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.390358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 863/1733] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:42.402986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:42.411641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.416544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 871/1734] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:42.419850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:42.425584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.463241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02125cf-cb31-4036-9944-a9ab52ae31c7 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.590567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e57f9-3f32-45c2-8851-c59a13780f74 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.599000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.599855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.600336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.601441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.602651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.606152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.614195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.614813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.615089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.615357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.637948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.640240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.653497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 864/1735] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:42.655886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.681777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f2cc5-2a86-492c-ae3c-7ab567e1267a service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.687762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.689474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.689755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.690228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.691061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.692846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.757030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 872/1736] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/projects => generated 405 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:42.760643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.860407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 865/1737] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/users => generated 290 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:42.864634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.908098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dedf9-da29-4fca-be09-eb85edbecac6 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.910567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aafb3-b51b-4e28-8dcd-30d8e7d929a7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:42.920543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.921197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.921474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.921880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.922536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.937774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.942432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:42.943080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:42.943365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:42.943688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:42.950672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:42.960566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:42.964507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 866/1738] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:42.968948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:42.998706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c200aac-1581-4be3-8f34-cc371edef82c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.015685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.016358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.016440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.017216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] PATH_INFO: `/v3/projects/1d35a64e5cd7423ba642529e822d1342/users/b93cfec912da4b958954b601b35c6db5/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.070182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d35a64e5cd7423ba642529e822d1342, user_id=b93cfec912da4b958954b601b35c6db5, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.072875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.104072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 873/1739] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:42 2022] POST /identity/v3/users => generated 303 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:55:43.108462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.128417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595831d-c2dc-4756-9cd0-7216fd4125fd admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.170422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.171061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.171333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.171837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.172508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.174278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.195946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 874/1740] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:55:43.196948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.199555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 867/1741] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:42 2022] PUT /identity/v3/projects/1d35a64e5cd7423ba642529e822d1342/users/b93cfec912da4b958954b601b35c6db5/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:43.203833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.241485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8362530-1f38-481d-bc4d-288c9b980980 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.344244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958e344-9102-415d-a43f-b1d70e8aead9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.354439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.355716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.356148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.357051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.358009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.359834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.362940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.363572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.363854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.364124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.364816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.366579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.376984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 868/1742] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:43.385304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.388226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 875/1743] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:43.389217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.400365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d451ab-9ad7-4740-8344-0ca1ddb27eee admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.410184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.410877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.411160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.411541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] PATH_INFO: `/v3/projects/f50fd53dc66642daae636e9640d4085c/users/ae3cbbe6ff5c43e88f1881e13043c407/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.431832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba29933-fb6b-4bff-826d-8a041db2022c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.457971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.463358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.463358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.463358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] PATH_INFO: `/v3/projects/1d35a64e5cd7423ba642529e822d1342/users/b93cfec912da4b958954b601b35c6db5/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.464192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50fd53dc66642daae636e9640d4085c, user_id=ae3cbbe6ff5c43e88f1881e13043c407, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.467393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d35a64e5cd7423ba642529e822d1342, user_id=b93cfec912da4b958954b601b35c6db5, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.469285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b157b-0880-40ab-8cd0-7e1a45765b4c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.471641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988d89c-25be-4d45-a487-4207dd832686 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.523666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 869/1744] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:43 2022] PUT /identity/v3/projects/1d35a64e5cd7423ba642529e822d1342/users/b93cfec912da4b958954b601b35c6db5/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:43.525795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1da6-822c-48b7-94e4-0d4bdd65d680 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.525795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1da6-822c-48b7-94e4-0d4bdd65d680 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.525795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc1da6-822c-48b7-94e4-0d4bdd65d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.585645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 876/1745] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:43 2022] PUT /identity/v3/projects/f50fd53dc66642daae636e9640d4085c/users/ae3cbbe6ff5c43e88f1881e13043c407/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:43.601087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386ec2b-a807-49b3-a871-4a059bae168d None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.601866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386ec2b-a807-49b3-a871-4a059bae168d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.602124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386ec2b-a807-49b3-a871-4a059bae168d None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.605323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-02cc1da6-822c-48b7-94e4-0d4bdd65d680 None None] MFA Rules not processed for user `b93cfec912da4b958954b601b35c6db5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:43.606964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 877/1746] 10.222.0.72 () {54 vars in 961 bytes} [Wed Aug 24 07:55:43 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:43.608524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.608807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-02cc1da6-822c-48b7-94e4-0d4bdd65d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.609868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.610152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.682268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] MFA Rules not processed for user `ae3cbbe6ff5c43e88f1881e13043c407`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:43.684238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.736916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:43.768522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 870/1747] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:43.776316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e209-e720-4a62-a83a-857020aa3039 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.776604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e209-e720-4a62-a83a-857020aa3039 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.777136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08e209-e720-4a62-a83a-857020aa3039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.799343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 878/1748] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:43.800420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:43.800871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-bf08e209-e720-4a62-a83a-857020aa3039 None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:43.802408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.804790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bf08e209-e720-4a62-a83a-857020aa3039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.824812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:43.919345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60642765-0485-4692-8aca-bf4d44b60ffe None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:43.926450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:43.927323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.927553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.927965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.930048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.933374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 871/1749] 10.222.0.72 () {58 vars in 1058 bytes} [Wed Aug 24 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:43.935035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:43.935327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:43.935597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:43.948263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:43.950383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:43.972665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] MFA Rules not processed for user `51729e82a3784d04aef89ed83f1576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:43.976601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:43.979238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 879/1750] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:43.981822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.132430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125247b-3538-464e-a473-fd8819a86ea7 service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.139043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 872/1751] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:44.139043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.164823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.167568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.167890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.168187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.204224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.208239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.247041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 880/1752] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:44.248111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.252777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc9a4-f503-44b8-a17c-4e80cf74b5f6 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.257471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.258351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.260598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.260598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.260598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.264466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.300528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794cfd5-5874-4ae7-9f35-1638c40e5753 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.317916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.318637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.318831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.319174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.338656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.340926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.366381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 881/1753] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:44.403895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 873/1754] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:44 2022] POST /identity/v3/projects => generated 403 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:44.405105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.424538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.433965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:44.437244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2ed68-d3f3-4b0e-a6a7-88f423d21f8e service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.625121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5172da-4d3b-4ad7-a3f8-9ecb4dd73697 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.629601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.630388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.631007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.631306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.643502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.644745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.645205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.645682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.646328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.656502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.748705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.750350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.761781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 874/1755] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:44.794628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 882/1756] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:44 2022] POST /identity/v3/users => generated 308 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:44.801034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.823483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532cc2f-820e-4bab-8dbe-19348186f740 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.829965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.833333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.833333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.833333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.833333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.834496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:44.850832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 875/1757] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:44.854083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:44.869456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35635140-923d-4c6a-86f5-de71f48a4a7d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:44.915076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:44.915076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:44.915076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:44.915076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] PATH_INFO: `/v3/projects/412d2147bbb246e4b2667d7f8465689c/users/522324097c1a468f8030a0fc84f61514/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:44.970284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412d2147bbb246e4b2667d7f8465689c, user_id=522324097c1a468f8030a0fc84f61514, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:44.975723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:45.129895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 883/1758] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:44 2022] PUT /identity/v3/projects/412d2147bbb246e4b2667d7f8465689c/users/522324097c1a468f8030a0fc84f61514/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:45.133562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:45.145895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9eafd7-60fa-4484-8a57-7ac17febfc1a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:45.408873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:45.409798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:45.410228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:45.410646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:45.412858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:45.414987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e4869-a73f-4ea8-ba2b-2aa73b011801 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:45.469215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:45.503977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6e1f-7d21-42f7-bacd-eb1f7b396f77 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:45.519092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 876/1759] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:45.533695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:45.534402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:45.534750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:45.536569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] PATH_INFO: `/v3/projects/412d2147bbb246e4b2667d7f8465689c/users/522324097c1a468f8030a0fc84f61514/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:45.552612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412d2147bbb246e4b2667d7f8465689c, user_id=522324097c1a468f8030a0fc84f61514, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:45.555644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:45.611191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:45.611488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:45.611772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:45.614668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 884/1760] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:45 2022] PUT /identity/v3/projects/412d2147bbb246e4b2667d7f8465689c/users/522324097c1a468f8030a0fc84f61514/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:45.691818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] MFA Rules not processed for user `522324097c1a468f8030a0fc84f61514`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:45.693651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:45.697747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:45.937519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 877/1761] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:46.544381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:46.666294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef852e-0f37-4d39-a196-c31a1b5fb907 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:46.671067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:46.677098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:46.677098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:46.677098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:46.690433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:46.692061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae397472-8bfd-4f91-9d12-b0b392da9b3a service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:46.723940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 885/1762] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:46 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:46.983136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:47.308841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:47.396936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70cf23-2c0f-4ab4-a1c5-fa68a01e8783 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:47.402686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:47.403639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:47.404119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:47.406488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] PATH_INFO: `/v3/users/ae3cbbe6ff5c43e88f1881e13043c407` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:47.407918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3cbbe6ff5c43e88f1881e13043c407)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:47.412468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:47.676480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:47.745115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3cbbe6ff5c43e88f1881e13043c407'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:47.745836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3cbbe6ff5c43e88f1881e13043c407'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:47.754316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3cbbe6ff5c43e88f1881e13043c407'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:47.841376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3cbbe6ff5c43e88f1881e13043c407'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:47.857099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3cbbe6ff5c43e88f1881e13043c407'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:47.865050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 878/1763] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:47 2022] DELETE /identity/v3/users/ae3cbbe6ff5c43e88f1881e13043c407 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:47.877698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:47.877698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:47.877698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:47.902676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:47.903910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:47.999297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 886/1764] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:48.011001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:48.013762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:48.108313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaeb2f-b231-4106-be51-ddfc6fe2f387 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:48.124729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:48.125998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:48.126310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:48.126615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:48.137228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:48.138667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:48.148721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 879/1765] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:48.662259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:48.666386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:48.697148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:48.706891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dfa00-20a7-4943-a2b1-ee4022564e9c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:48.720926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:48.721670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:48.721952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:48.722225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] PATH_INFO: `/v3/projects/f50fd53dc66642daae636e9640d4085c` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:48.727751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50fd53dc66642daae636e9640d4085c)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:48.734577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:48.938412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50fd53dc66642daae636e9640d4085c', 'request_id': 'req-66b268a9-917b-4a3a-b532-956b55b3bd5f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:48.947008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50fd53dc66642daae636e9640d4085c', 'request_id': 'req-66b268a9-917b-4a3a-b532-956b55b3bd5f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:48.953978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50fd53dc66642daae636e9640d4085c', 'request_id': 'req-66b268a9-917b-4a3a-b532-956b55b3bd5f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.088888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] The token cache is being invalidate because project f50fd53dc66642daae636e9640d4085c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:49.089154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.093167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-66b268a9-917b-4a3a-b532-956b55b3bd5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.094765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 887/1766] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:48 2022] DELETE /identity/v3/projects/f50fd53dc66642daae636e9640d4085c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:49.098977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:49.112752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178cf6-7971-4ba4-80aa-5ee985fb84d0 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:49.201114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:49.202097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:49.202567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:49.203378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] PATH_INFO: `/v3/users/522324097c1a468f8030a0fc84f61514` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:49.204815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=522324097c1a468f8030a0fc84f61514)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:49.207266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:49.478254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522324097c1a468f8030a0fc84f61514'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.479134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522324097c1a468f8030a0fc84f61514'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.507700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522324097c1a468f8030a0fc84f61514'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.632277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522324097c1a468f8030a0fc84f61514'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.657989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522324097c1a468f8030a0fc84f61514'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:49.690908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 880/1767] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:49 2022] DELETE /identity/v3/users/522324097c1a468f8030a0fc84f61514 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:49.796216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:49.796318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:49.796596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:49.937200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:49.939924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:50.179599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 888/1768] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:50.274545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:50.434467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:50.640590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:50.642352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:50.734035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:50.735971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:50.773765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e07f0e-67fd-47f0-993f-96f1b2f68c9d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:50.789511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:50.791200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:50.791556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:50.792642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:50.806776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:50.808809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bc514-6ca9-4849-b59a-47613e19966f service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:50.837681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 881/1769] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:55:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:55:50.851154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425cab1f-a7f3-46d0-9d1c-633e6c06f93c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:50.862826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:50.863948 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:50.864481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:50.864990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] PATH_INFO: `/v3/projects/412d2147bbb246e4b2667d7f8465689c` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:50.866537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=412d2147bbb246e4b2667d7f8465689c)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:50.869513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:51.126642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412d2147bbb246e4b2667d7f8465689c', 'request_id': 'req-171b949b-7fbe-4109-a614-1f48fef54a5d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.148499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412d2147bbb246e4b2667d7f8465689c', 'request_id': 'req-171b949b-7fbe-4109-a614-1f48fef54a5d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.162720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412d2147bbb246e4b2667d7f8465689c', 'request_id': 'req-171b949b-7fbe-4109-a614-1f48fef54a5d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.310237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] The token cache is being invalidate because project 412d2147bbb246e4b2667d7f8465689c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:51.310742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.312766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.314928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 889/1770] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:50 2022] DELETE /identity/v3/projects/412d2147bbb246e4b2667d7f8465689c => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:51.320322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d238-5346-4a79-ad7f-8edc698c415d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:51.322223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d238-5346-4a79-ad7f-8edc698c415d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:51.323018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44d238-5346-4a79-ad7f-8edc698c415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:51.409636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2f44d238-5346-4a79-ad7f-8edc698c415d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:51.409636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2f44d238-5346-4a79-ad7f-8edc698c415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:51.494295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:51.499362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:51.643456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 882/1771] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:51.654915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b949b-7fbe-4109-a614-1f48fef54a5d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:51.655021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602fcd-38dd-4385-a01d-2bce7ead646b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:51.655125 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:51.655224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602fcd-38dd-4385-a01d-2bce7ead646b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:51.656319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:51.656863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:51.657377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] PATH_INFO: `/v3/users/fd3e795dcd5442e094863cb3200fea58` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:51.660011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd3e795dcd5442e094863cb3200fea58)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:51.660602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee602fcd-38dd-4385-a01d-2bce7ead646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:51.662593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:51.725787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ee602fcd-38dd-4385-a01d-2bce7ead646b None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:51.728257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ee602fcd-38dd-4385-a01d-2bce7ead646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:51.867002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 883/1772] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 24 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:51.868896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:51.869658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:51.872236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:51.916981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:51.919118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:51.945021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 884/1773] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:51.945021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:51.948230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:51.964975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:51.981802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606031ca-2c0f-4a45-bd44-4a77773d29f2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:51.984750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd3e795dcd5442e094863cb3200fea58'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:51.997580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd3e795dcd5442e094863cb3200fea58'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.002053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd3e795dcd5442e094863cb3200fea58'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.002053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd3e795dcd5442e094863cb3200fea58'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.015047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.015717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.015996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.016267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] PATH_INFO: `/v3/users/fd45688f32194dc59fdef259b1cf81f3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.019993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd45688f32194dc59fdef259b1cf81f3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.022015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.145272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd3e795dcd5442e094863cb3200fea58'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.181329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:52.198751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 890/1774] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:51 2022] DELETE /identity/v3/users/fd3e795dcd5442e094863cb3200fea58 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:52.200808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.202415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.349355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd45688f32194dc59fdef259b1cf81f3'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.349355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd45688f32194dc59fdef259b1cf81f3'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.349355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd45688f32194dc59fdef259b1cf81f3'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.598332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e95ed-7f30-4ba1-810e-6121fae4aa89 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.598332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.598332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.598332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 891/1775] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:52.599596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.600369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.600369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcb32d-7e35-4c2b-ba30-e80291c5c8e3 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.600369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.600689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2e35-4ee1-4b77-88b3-a38a6824e89d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 892/1776] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:52.601160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.601770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 893/1777] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:52.601770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.601770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.617938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74ab3d-ae44-462b-a0b7-3b13f9e22357 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.623046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.677006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 894/1778] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:52.677753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.677753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:52.677753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd66fba-3095-4968-871f-0c618cb59898 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.677994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.677994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.677994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.677994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.683477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.685619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.701774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 895/1779] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:52.702984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.726084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f1a98-8613-4de3-9976-5007688c4317 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.729387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd45688f32194dc59fdef259b1cf81f3'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.779095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd45688f32194dc59fdef259b1cf81f3'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:52.779674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.779674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.779674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.779674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:52.779991 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:52.780071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 885/1780] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:51 2022] DELETE /identity/v3/users/fd45688f32194dc59fdef259b1cf81f3 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:52.780156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:52.780241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:52.882330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbf1a7-0e62-4471-8c14-dc903e86ab21 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:52.892762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:52.893684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:52.894136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:52.894567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.017488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:53.019617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae6a5d-2dc7-47f1-9888-d3148541f8ca service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:53.032017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 886/1781] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:53.033836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.034282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.034778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.066892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:53.069095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:53.082755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 887/1782] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:53.084127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:53.086172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:53.127002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0fc7d-ac4a-411b-8217-6e8cf98d72ef None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:53.142988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:53.146822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.147322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.147901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] PATH_INFO: `/v3/users/550a229c588747438329b603b37b4764` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.151204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] RBAC: Authorizing `identity:delete_user(user_id=550a229c588747438329b603b37b4764)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:53.158932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:53.325763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 896/1783] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:52 2022] POST /identity/v3/projects => generated 405 bytes in 614 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:53.325763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:53.489899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd8894-ff38-46fa-8a9e-0d5c4ef5a003 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:53.497589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:53.498608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.499111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.499775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.575381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550a229c588747438329b603b37b4764'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:53.576108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550a229c588747438329b603b37b4764'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:53.588682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550a229c588747438329b603b37b4764'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:53.625042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:53.627358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:53.647408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 897/1784] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:53.648409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:53.663341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf32529-6817-4cd6-a41a-976d8f28cba8 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:53.678204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:53.680763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.680763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.682400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.685095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:53.688643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550a229c588747438329b603b37b4764'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:53.689555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:53.719131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-14977b60-acb5-47c9-989e-9660cd06358f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550a229c588747438329b603b37b4764'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:53.749697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 888/1785] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:53 2022] DELETE /identity/v3/users/550a229c588747438329b603b37b4764 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:53.764101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.765617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.766126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.809942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:53.811649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:53.870037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 898/1786] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:53 2022] POST /identity/v3/users => generated 303 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:53.874557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:53.877635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:53.921716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8849e-15f5-4313-a02c-2c689b92ea52 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:53.944300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 889/1787] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:53.960754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:53.962606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:53.962606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:53.962606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:53.963468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:53.965448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:53.977276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 899/1788] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:53.984161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:54.005912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded0f56-2eca-427e-9e7f-8b7c99725e55 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:54.030490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:54.033104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.033104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.033104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.033104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:54.045389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:54.071225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 890/1789] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:54.075249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:54.093705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf386f-d64e-4926-997a-e18de3b307db admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:54.096237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:54.097576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.097994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.098525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] PATH_INFO: `/v3/projects/3b86ad2835c149b188a7448d8eed4746/users/4083262ddb834617855e19aad3c1e177/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.124586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:54.127475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:54.131248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b86ad2835c149b188a7448d8eed4746, user_id=4083262ddb834617855e19aad3c1e177, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:54.144324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96168ac8-93db-436e-ba20-e9d15898b876 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:54.226665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228b5f-31cc-4b15-b7f4-68705b174a3c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:54.230416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:54.235150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:54.237104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.237104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.237104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.237970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 900/1790] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:54 2022] PUT /identity/v3/projects/3b86ad2835c149b188a7448d8eed4746/users/4083262ddb834617855e19aad3c1e177/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:54.244151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a3249-069d-4d1c-9e90-3da170e89359 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.244798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a3249-069d-4d1c-9e90-3da170e89359 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.245248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a3249-069d-4d1c-9e90-3da170e89359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.273036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:54.279106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:54.307806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-294a3249-069d-4d1c-9e90-3da170e89359 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:54.312638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 891/1791] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:54.313148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-294a3249-069d-4d1c-9e90-3da170e89359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:54.390018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:54.432992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 901/1792] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:54.468589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:54.479382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f352a1-e994-4a77-b9bf-3f14f59d377f service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:54.591719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:54.593362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.593902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.594475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.614247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:54.623852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9db82-6aaa-467b-bbe7-0ec60a069292 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:54.642879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 892/1793] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:54.891318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfee84-bb0d-4259-95d7-3feba6fa90c8 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.891964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfee84-bb0d-4259-95d7-3feba6fa90c8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.892482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfee84-bb0d-4259-95d7-3feba6fa90c8 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:54.894939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 902/1794] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:54.957125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:54.957507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:54.957787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:55.039598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:55.039598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:55.214899 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 893/1795] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:55:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:55.490612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886a82-f480-4d92-9210-238f34ef12e1 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:55.491176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886a82-f480-4d92-9210-238f34ef12e1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:55.491176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14886a82-f480-4d92-9210-238f34ef12e1 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:55.492435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 903/1796] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:55:55.503643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:55.505320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:55.529871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:55.531219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:55.531849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:55.544619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850be233-6e0d-4cf4-8d22-2603700a57fd None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:55.552072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:55.552949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:55.553333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:55.553752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] PATH_INFO: `/v3/projects/d2a4b5afd95d436f916d9cc485d77b79` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:55.555466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a4b5afd95d436f916d9cc485d77b79)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:55.559232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:55.572023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:55.625713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:55.630878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:55.660579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 904/1797] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:55:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:55:55.894688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a4b5afd95d436f916d9cc485d77b79', 'request_id': 'req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:55.956928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:55.959022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:56.021939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a4b5afd95d436f916d9cc485d77b79', 'request_id': 'req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.063518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a4b5afd95d436f916d9cc485d77b79', 'request_id': 'req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.065413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b479ef-e240-4376-a205-bc7ddccd93c9 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:56.120171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] The token cache is being invalidate because project d2a4b5afd95d436f916d9cc485d77b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:56.143641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.145138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.147851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 894/1798] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:55 2022] DELETE /identity/v3/projects/d2a4b5afd95d436f916d9cc485d77b79 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:56.152329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:56.155113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:56.264581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:56.274349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:56.275026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:56.275374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:56.275599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] PATH_INFO: `/v3/projects/70557f70665b440ba434d06d57b04f78` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:56.290353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70557f70665b440ba434d06d57b04f78)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:56.290353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:56.395819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fbd15-76ac-4ee0-a00a-da9e0412d10d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:56.418364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:56.418364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:56.418364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:56.418364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:56.421195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:56.442947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:56.447123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:56.463056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 895/1799] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:56.464627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:56.466353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:56.485208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2ed80-dff7-4d53-8311-14f5d1c29b8c service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:56.496782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70557f70665b440ba434d06d57b04f78', 'request_id': 'req-17538032-c8cf-42c5-9a53-4f178a653e84'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.537201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70557f70665b440ba434d06d57b04f78', 'request_id': 'req-17538032-c8cf-42c5-9a53-4f178a653e84'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.558166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70557f70665b440ba434d06d57b04f78', 'request_id': 'req-17538032-c8cf-42c5-9a53-4f178a653e84'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.607486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:56.608186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:56.608465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:56.608834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] PATH_INFO: `/v3/projects/e95df705878f4d9bbcec5d24e0125e58` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:56.610039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95df705878f4d9bbcec5d24e0125e58)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:56.611986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:56.737255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:56.788505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] The token cache is being invalidate because project 70557f70665b440ba434d06d57b04f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:56.788505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.789840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.800456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 905/1800] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:55 2022] DELETE /identity/v3/projects/70557f70665b440ba434d06d57b04f78 => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:55:56.801567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:56.803227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:56.826436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17538032-c8cf-42c5-9a53-4f178a653e84 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:56.832307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:56.837812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:56.839816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:56.840158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] PATH_INFO: `/v3/users/62ef39d3e90d46afb6bfdea7a50437c7` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:56.842331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ef39d3e90d46afb6bfdea7a50437c7)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:56.851184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:56.856866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95df705878f4d9bbcec5d24e0125e58', 'request_id': 'req-521f49f1-8319-496d-b09f-673785270bc6'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.934043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95df705878f4d9bbcec5d24e0125e58', 'request_id': 'req-521f49f1-8319-496d-b09f-673785270bc6'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.945869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95df705878f4d9bbcec5d24e0125e58', 'request_id': 'req-521f49f1-8319-496d-b09f-673785270bc6'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.972512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] The token cache is being invalidate because project e95df705878f4d9bbcec5d24e0125e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:56.972861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.980267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:56.984895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 896/1801] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:56 2022] DELETE /identity/v3/projects/e95df705878f4d9bbcec5d24e0125e58 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:56.984895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:56.984895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:57.125807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f49f1-8319-496d-b09f-673785270bc6 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:57.132207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:57.145847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:57.146362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:57.147069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:57.153036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:57.170128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ef39d3e90d46afb6bfdea7a50437c7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.182396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ef39d3e90d46afb6bfdea7a50437c7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.187107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ef39d3e90d46afb6bfdea7a50437c7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.187107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ef39d3e90d46afb6bfdea7a50437c7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.189501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:57.196214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:57.227557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 897/1802] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:57.228528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:57.235943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:57.294403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ef39d3e90d46afb6bfdea7a50437c7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.374179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 906/1803] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:56 2022] DELETE /identity/v3/users/62ef39d3e90d46afb6bfdea7a50437c7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:57.382928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:57.388763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:57.430031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092a162-2f67-4285-8d93-bb4569f763f1 service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:57.499922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:57.501052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:57.501486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:57.502013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] PATH_INFO: `/v3/users/28d670dfd1994727ad9eafa33e7019ee` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:57.503918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d670dfd1994727ad9eafa33e7019ee)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:57.506781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:57.549658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33bd24-3215-44bb-ad66-97fd73dc06fe admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:57.575861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:57.588671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:57.589476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:57.589947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] PATH_INFO: `/v3/users/b93cfec912da4b958954b601b35c6db5` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:57.591561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93cfec912da4b958954b601b35c6db5)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:57.594507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:57.793799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d670dfd1994727ad9eafa33e7019ee'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.813219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d670dfd1994727ad9eafa33e7019ee'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.833781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d670dfd1994727ad9eafa33e7019ee'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.882383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93cfec912da4b958954b601b35c6db5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.896144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93cfec912da4b958954b601b35c6db5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.897829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93cfec912da4b958954b601b35c6db5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.898713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93cfec912da4b958954b601b35c6db5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.970601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d670dfd1994727ad9eafa33e7019ee'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:57.992292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d670dfd1994727ad9eafa33e7019ee'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.005001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 898/1804] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:57 2022] DELETE /identity/v3/users/28d670dfd1994727ad9eafa33e7019ee => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:58.005942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:58.007762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:58.056476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d72d62d5-5e4b-4ee7-9a85-86f4b7a45bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93cfec912da4b958954b601b35c6db5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.138977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed395d-15e4-4d1b-9bdc-5dff715539ca admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:58.145548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 907/1805] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:57 2022] DELETE /identity/v3/users/b93cfec912da4b958954b601b35c6db5 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:58.145548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:58.145548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:58.145548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:58.182024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:58.184763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:58.279318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:58.280056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:58.280337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:58.281232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] PATH_INFO: `/v3/users/fd98dddf597d4355822346ac9bb59ae8` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:58.284041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd98dddf597d4355822346ac9bb59ae8)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:58.288444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:58.310217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 908/1806] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:58.310217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:58.310217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:58.336068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4860f-5b0e-47f6-8085-7a0e65fe42d9 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:58.342532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:58.343578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:58.344027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:58.344507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] PATH_INFO: `/v3/projects/46f5b7ef37a142e6879ca69dba9a9560` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:58.346259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f5b7ef37a142e6879ca69dba9a9560)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:58.349163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:58.462512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:58.549371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f5b7ef37a142e6879ca69dba9a9560', 'request_id': 'req-be1c67d5-7191-424c-a8fa-454388da8236'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.552878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd98dddf597d4355822346ac9bb59ae8'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.553831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd98dddf597d4355822346ac9bb59ae8'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.570900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f5b7ef37a142e6879ca69dba9a9560', 'request_id': 'req-be1c67d5-7191-424c-a8fa-454388da8236'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.572155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd98dddf597d4355822346ac9bb59ae8'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.589672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f5b7ef37a142e6879ca69dba9a9560', 'request_id': 'req-be1c67d5-7191-424c-a8fa-454388da8236'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.666059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd98dddf597d4355822346ac9bb59ae8'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.697652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b84fd392-d95b-4ea9-90aa-e8302d81fa61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd98dddf597d4355822346ac9bb59ae8'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.725307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 899/1807] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:55:58 2022] DELETE /identity/v3/users/fd98dddf597d4355822346ac9bb59ae8 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:58.726969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b85c2-e46b-43ae-9d2d-0665390bcba2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:58.727390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b85c2-e46b-43ae-9d2d-0665390bcba2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:58.728064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b85c2-e46b-43ae-9d2d-0665390bcba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:58.737103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] The token cache is being invalidate because project 46f5b7ef37a142e6879ca69dba9a9560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:55:58.737622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.739506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:55:58.741941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 909/1808] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:55:58 2022] DELETE /identity/v3/projects/46f5b7ef37a142e6879ca69dba9a9560 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:58.742973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:58.748472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:58.819865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f11b85c2-e46b-43ae-9d2d-0665390bcba2 None None] MFA Rules not processed for user `4083262ddb834617855e19aad3c1e177`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:58.822356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f11b85c2-e46b-43ae-9d2d-0665390bcba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:58.872437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c67d5-7191-424c-a8fa-454388da8236 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:58.978736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 900/1809] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:58.980988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:58.981283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:58.981559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.029341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:59.031289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.031423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:59.031514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.031737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.032151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.034238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:59.111480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 901/1810] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:59.111480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.113327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:59.138263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207c4df-ec86-4005-8e1b-fd816cad90d7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.144628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.146009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.146009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.146341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.149223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.153279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d095023-cc91-4899-a7a4-3478b50999f2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.247386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.249545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.277389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 902/1811] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:59 2022] POST /identity/v3/projects => generated 405 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:59.279779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.280676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.280676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.303746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 910/1812] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:55:59.305958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.342203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe18d3-adcb-4cba-9cfa-3cdefff58a30 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.347465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:55:59.349028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:55:59.523908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.523908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.523908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.523908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.525874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.525874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.527427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:55:59.539320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 903/1813] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:55:59.541760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.557021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51c0e-381b-4be9-887b-90feb2d14846 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.571890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.575110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.575869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.576730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.580112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:55:59.581153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.584128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.604193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 904/1814] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:55:59.606859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.619285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 911/1815] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:59 2022] POST /identity/v3/users => generated 309 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:59.622739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.641316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7fa4-00c0-4eeb-809a-f55dcee42abf admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.643253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9916da2-12ca-4554-b690-a63c494f16c4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.650038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.651776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.652632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.654543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.657487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.658222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.661088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.661631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.662093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.663204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.663416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.677484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.694096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 912/1816] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:55:59.695943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.719856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d7f44-ba54-4dc1-9c81-b6059bd68a52 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:55:59.726198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:55:59.727245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:55:59.727863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:55:59.728558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] PATH_INFO: `/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1/users/1831b40189e84a729621441451cb6a3c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:55:59.764995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752bd72aaa0f47b6a4c0b611e9ba59e1, user_id=1831b40189e84a729621441451cb6a3c, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:55:59.769024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:55:59.880682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 905/1817] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:55:59 2022] POST /identity/v3/projects => generated 379 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:55:59.884697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:55:59.906833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 913/1818] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:55:59 2022] PUT /identity/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1/users/1831b40189e84a729621441451cb6a3c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:55:59.909775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.079165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dd068-6919-4219-acf6-05619613a09b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.087780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a6e74-4fda-42ac-bd7a-341483c78e58 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.106785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.113386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.113386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.113386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.113386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.113775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.113775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.113775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.113775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.117436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.121959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.125413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.148738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 914/1819] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:00.150887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.154420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:00.173988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cd4e8-afb4-4867-80fb-c0825eada85b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.181119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.182090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.182580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.182996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] PATH_INFO: `/v3/projects/1d35a64e5cd7423ba642529e822d1342` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.184273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d35a64e5cd7423ba642529e822d1342)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.186511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.323887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 906/1820] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:55:59 2022] POST /identity/v3/users => generated 290 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:00.323887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.357930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eba9a5-960f-4000-b75f-989a146eef27 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.365000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.366800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.367315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.368093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] PATH_INFO: `/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1/users/1831b40189e84a729621441451cb6a3c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.371773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=752bd72aaa0f47b6a4c0b611e9ba59e1, user_id=1831b40189e84a729621441451cb6a3c, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.375609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.422239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 907/1821] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:00 2022] PUT /identity/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1/users/1831b40189e84a729621441451cb6a3c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:00.426662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.434175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d35a64e5cd7423ba642529e822d1342', 'request_id': 'req-770bf8e7-d903-48f7-a6f6-954914e2a3c5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:00.455745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d35a64e5cd7423ba642529e822d1342', 'request_id': 'req-770bf8e7-d903-48f7-a6f6-954914e2a3c5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:00.469958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d35a64e5cd7423ba642529e822d1342', 'request_id': 'req-770bf8e7-d903-48f7-a6f6-954914e2a3c5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:00.547338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a7c90-2357-436d-801f-b2502075f6bf admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.554810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.555982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.556608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.557203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.558764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.561988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.578015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 908/1822] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:00.584264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] The token cache is being invalidate because project 1d35a64e5cd7423ba642529e822d1342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:00.584820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:00.585517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-770bf8e7-d903-48f7-a6f6-954914e2a3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:00.591847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 915/1823] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:00 2022] DELETE /identity/v3/projects/1d35a64e5cd7423ba642529e822d1342 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:00.607586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.610471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b497da-b297-4c32-85ec-606e2ef2f623 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.611099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b497da-b297-4c32-85ec-606e2ef2f623 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.611690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b497da-b297-4c32-85ec-606e2ef2f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.613391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:00.652710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-92b497da-b297-4c32-85ec-606e2ef2f623 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:00.655154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2300d2-634b-474d-af70-05bbe6c17ff7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.656448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-92b497da-b297-4c32-85ec-606e2ef2f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:00.663746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.665013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.665013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.665013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.665775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.668125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.697742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 916/1824] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:00.703731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 909/1825] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:00.708341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.708689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.708937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.713038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.743941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c26ab-a990-417b-9286-da0ac8cff5e1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.765319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.767929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.767929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.767929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] PATH_INFO: `/v3/projects/b91ca21043ba41afa802db4d2523b774/users/57db7a0affef431c8d1ec206cf2b28d0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.775010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:00.776739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:00.798476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 917/1826] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:00.799817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:00.801651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:00.810522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b91ca21043ba41afa802db4d2523b774, user_id=57db7a0affef431c8d1ec206cf2b28d0, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.813982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.839307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601cc802-806d-4e08-814b-8ddb03bab022 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:00.852231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:00.853258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:00.854076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:00.855292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] PATH_INFO: `/v3/projects/6a217e5dacbf48dd8545bf4a68444c59` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:00.856947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a217e5dacbf48dd8545bf4a68444c59)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:00.860402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:00.936405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 910/1827] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:00 2022] PUT /identity/v3/projects/b91ca21043ba41afa802db4d2523b774/users/57db7a0affef431c8d1ec206cf2b28d0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:00.936405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.057050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6d9dc-6f74-4f43-abdb-998b13d1e95d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:01.062824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:01.063634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.064089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.064511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.065338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:01.065795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:01.067553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787823ed-280e-48be-beb7-b351362e1279 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:01.084044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 911/1828] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:01.085505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.085886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.086259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.121884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a217e5dacbf48dd8545bf4a68444c59', 'request_id': 'req-2e838ff6-71bc-4920-9991-ebcc86502178'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.131781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] MFA Rules not processed for user `57db7a0affef431c8d1ec206cf2b28d0`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:01.133421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.148405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a217e5dacbf48dd8545bf4a68444c59', 'request_id': 'req-2e838ff6-71bc-4920-9991-ebcc86502178'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.156941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a217e5dacbf48dd8545bf4a68444c59', 'request_id': 'req-2e838ff6-71bc-4920-9991-ebcc86502178'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.244743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 912/1829] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:01.250549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.274224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.354145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f754cdf-1f5d-4893-a8b7-712bdac991e7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:01.358149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] The token cache is being invalidate because project 6a217e5dacbf48dd8545bf4a68444c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:01.358718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.360797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.360912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:01.360912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.361199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.361454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] PATH_INFO: `/v3/projects/a0132d93372a452d8b399dd12d16c2e1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.362930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0132d93372a452d8b399dd12d16c2e1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:01.364675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 918/1830] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:00 2022] DELETE /identity/v3/projects/6a217e5dacbf48dd8545bf4a68444c59 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:01.365209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:01.365747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.367323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.388331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e838ff6-71bc-4920-9991-ebcc86502178 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:01.394919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:01.395551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.395832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.396101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.396972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:01.399297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:01.521521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 919/1831] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:01 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:01.522574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.524431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.573084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0132d93372a452d8b399dd12d16c2e1', 'request_id': 'req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.608832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c5e54-2934-429b-86dc-b484e3eea233 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:01.693565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:01.694209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.694728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.694728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.695352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:01.697391 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:01.763044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0132d93372a452d8b399dd12d16c2e1', 'request_id': 'req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.775365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0132d93372a452d8b399dd12d16c2e1', 'request_id': 'req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.799339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] The token cache is being invalidate because project a0132d93372a452d8b399dd12d16c2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:01.799820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.800309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-064e722f-d5a5-4ec1-9b73-ed5c74ca2d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:01.805398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 913/1832] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:01 2022] DELETE /identity/v3/projects/a0132d93372a452d8b399dd12d16c2e1 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:01.805398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.805398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.805398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.826181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:01.829900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.842626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 914/1833] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:01.843732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.845276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:01.860292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46eb6b-6242-4424-9684-0cd39d8d0c55 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:01.880306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:01.880957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:01.881235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:01.881504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:01.883963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:01.888522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6840990-619b-4c5a-ae5a-05241c4fd36e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:01.902460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 920/1834] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:01 2022] POST /identity/v3/projects => generated 381 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:01.903704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:01.906107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.038261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 915/1835] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:01 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:02.040997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.040997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.040997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.054457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ec6e0-cd3b-43cb-a3ec-fb37e766dec7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.062038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:02.062544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.063557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.063673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.064170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.064827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] PATH_INFO: `/v3/users/fa46a12ba79f446da1936ec2e901d1c3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.069037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa46a12ba79f446da1936ec2e901d1c3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.075464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.092292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 916/1836] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:02.094230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.098202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.128615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73301cb1-66b1-4774-9909-cb8179b633fb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.138554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.139531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.139990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.140459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.141647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.145260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.159312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:02.235808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 917/1837] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:02 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:02.237911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.248125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777a58-b1c5-4371-871f-93cbb7511a65 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.255285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.261374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.261374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.261374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.261374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.261374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.285785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 918/1838] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:02.285785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.293729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b30d9-2335-4bf8-9923-2be0f57a6556 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.296845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa46a12ba79f446da1936ec2e901d1c3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:02.301433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.302092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.302368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.302651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.306135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:02.306422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.306995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa46a12ba79f446da1936ec2e901d1c3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:02.307866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa46a12ba79f446da1936ec2e901d1c3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:02.308526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa46a12ba79f446da1936ec2e901d1c3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:02.309386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.329584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 919/1839] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:02.330583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.332724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.354066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49acd-faf1-4228-841b-d69cb9789b93 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.365216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.365847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.371753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.371753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.371753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.371753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.380475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 920/1840] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:02.381747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.387251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.388319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa46a12ba79f446da1936ec2e901d1c3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:02.408225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 921/1841] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:01 2022] DELETE /identity/v3/users/fa46a12ba79f446da1936ec2e901d1c3 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:02.409285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.410666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:02.435056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cc349-235d-481e-b2b1-ccc550925148 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.534644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.535544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.536003 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.536366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.538331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.540239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.554987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f922d8-b1ef-4c4a-9399-5df8050764ac admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.562304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 921/1842] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:02.564166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.568516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.569537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.569963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.570289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.571336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.575912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.580204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6d77-74a7-4c2f-a26b-7b019a4bc01c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.588566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.589529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.589907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.590376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] PATH_INFO: `/v3/projects/9071337458c84bed954e6da88d460a31/users/963d819c41794194bf063fce8f25d99a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.628545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9071337458c84bed954e6da88d460a31, user_id=963d819c41794194bf063fce8f25d99a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.631731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.710319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 922/1843] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:02 2022] POST /identity/v3/projects => generated 389 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:02.713530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.764059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 922/1844] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:02 2022] PUT /identity/v3/projects/9071337458c84bed954e6da88d460a31/users/963d819c41794194bf063fce8f25d99a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:02.765012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:02.785710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b522dfd-1845-4389-a37e-fce0e66062d2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.861249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36843f0a-0b1e-4c44-9f68-d512a68605fb admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:02.866815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.867321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:02.867497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.867759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.868034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.868146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:02.868341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:02.868625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] PATH_INFO: `/v3/projects/02fc2813b8544abb8d31896f84e331f5/users/9b88b473ff024e088e009c7b1a9a1ada/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:02.868760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.871610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:02.885667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fc2813b8544abb8d31896f84e331f5, user_id=9b88b473ff024e088e009c7b1a9a1ada, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:02.888718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.184196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 923/1845] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:02 2022] PUT /identity/v3/projects/02fc2813b8544abb8d31896f84e331f5/users/9b88b473ff024e088e009c7b1a9a1ada/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:03.186176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.187449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 923/1846] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:02 2022] POST /identity/v3/users => generated 295 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:03.188677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.201004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45ec2f-8417-405f-9e84-d430643386bb admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.297424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5520de-bc5a-4b87-8d87-664b877bd459 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.304069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.304628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.305072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.305356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.306138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.308154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a25a7-7fab-4301-9322-0a39aea819e0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.308306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.309010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.309295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.309749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.310639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.312162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.319988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 924/1847] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:03.321368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25aa6a-f555-43e6-bf9f-87a30bbc2c46 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.321658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25aa6a-f555-43e6-bf9f-87a30bbc2c46 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.321945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25aa6a-f555-43e6-bf9f-87a30bbc2c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.332330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 924/1848] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:03.333383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.336634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.351210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ab6cc-f5e0-465a-97b0-7a5a5b766ca2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.357638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.362740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.362740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.362740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] PATH_INFO: `/v3/projects/30660254fd4c478f9013dcebba9b3d75` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.362740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30660254fd4c478f9013dcebba9b3d75)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.362740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.365030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-aa25aa6a-f555-43e6-bf9f-87a30bbc2c46 None None] MFA Rules not processed for user `9b88b473ff024e088e009c7b1a9a1ada`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:03.366301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aa25aa6a-f555-43e6-bf9f-87a30bbc2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.456392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:03.457159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 925/1849] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:03.458668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.459052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.459423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.500013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] MFA Rules not processed for user `1831b40189e84a729621441451cb6a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:03.501421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.510260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30660254fd4c478f9013dcebba9b3d75', 'request_id': 'req-0f47556b-6a0f-453c-bbfa-383f30f51566'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:03.575318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 926/1850] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:03.576072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.602675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30660254fd4c478f9013dcebba9b3d75', 'request_id': 'req-0f47556b-6a0f-453c-bbfa-383f30f51566'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:03.610208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30660254fd4c478f9013dcebba9b3d75', 'request_id': 'req-0f47556b-6a0f-453c-bbfa-383f30f51566'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:03.625564 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] The token cache is being invalidate because project 30660254fd4c478f9013dcebba9b3d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:03.625984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:03.627237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:03.634243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 925/1851] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:03 2022] DELETE /identity/v3/projects/30660254fd4c478f9013dcebba9b3d75 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:03.634243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.636300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.649522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46463b4a-a1a4-4a36-961b-bda1ffab0591 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.652384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47556b-6a0f-453c-bbfa-383f30f51566 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.658463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.658947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.659135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.659490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.659832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.660445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.660641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.661124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.661574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] PATH_INFO: `/v3/projects/9071337458c84bed954e6da88d460a31/users/963d819c41794194bf063fce8f25d99a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.663152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.665658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9071337458c84bed954e6da88d460a31, user_id=963d819c41794194bf063fce8f25d99a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.669854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69aec2-1085-42f4-840d-11340bfb7f4b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.676522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 926/1852] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:03.678098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.680471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.685900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 927/1853] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:03 2022] PUT /identity/v3/projects/9071337458c84bed954e6da88d460a31/users/963d819c41794194bf063fce8f25d99a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:03.691635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6b252-4845-43af-99da-39201879b5db None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.691635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6b252-4845-43af-99da-39201879b5db None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.691635 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6b252-4845-43af-99da-39201879b5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.694164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c164-d6c4-4fbf-b1a1-a380584c7280 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.736060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c8e6b252-4845-43af-99da-39201879b5db None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:03.737355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c8e6b252-4845-43af-99da-39201879b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.801794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.802421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.802762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.802972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.803613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.805644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb0f31-5a24-4ff1-ab31-d38fd2588c7b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.821455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 928/1854] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:03.822227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6b252-4845-43af-99da-39201879b5db None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:03.833260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6b252-4845-43af-99da-39201879b5db None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:03.838493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:03.846010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.846010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.846010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] PATH_INFO: `/v3/projects/83fc0ff07e764a789e40530a29b56af0/users/101c34e61a3144d8b5a7fe172bf5d5e1/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.856182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fc0ff07e764a789e40530a29b56af0, user_id=101c34e61a3144d8b5a7fe172bf5d5e1, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:03.858851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4184043-ac3c-4b6e-94f7-dfe27df9880e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:03.959253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 929/1855] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:03 2022] PUT /identity/v3/projects/83fc0ff07e764a789e40530a29b56af0/users/101c34e61a3144d8b5a7fe172bf5d5e1/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:03.960846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.961202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.961554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.974212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 927/1856] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/projects => generated 381 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:03.975647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee368cd-d9b0-4d95-89df-02c849394542 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:03.975930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee368cd-d9b0-4d95-89df-02c849394542 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:03.976205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee368cd-d9b0-4d95-89df-02c849394542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:03.996330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] MFA Rules not processed for user `963d819c41794194bf063fce8f25d99a`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:03.997709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:03.998907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-bee368cd-d9b0-4d95-89df-02c849394542 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:04.000291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bee368cd-d9b0-4d95-89df-02c849394542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:04.061853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 928/1857] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:04.063563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.064162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.064532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.078641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 930/1858] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:03 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:04.079845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.090735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bceb81-d259-4a5b-8813-35d3b1081df1 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.095969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.096568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.097411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.097691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.098332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.100454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.103243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] MFA Rules not processed for user `101c34e61a3144d8b5a7fe172bf5d5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:04.104632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:04.182044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 929/1859] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:04.183641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.194639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3e5f-b9bd-42e6-ab60-f6bfaf121c55 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.203375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 931/1860] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:04.203375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.203597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.203597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.203803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:04.203909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.203909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.203909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:04.203909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.204215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.221294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 930/1861] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:04.221294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.225411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:04.232146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106df63-c97e-47fb-b3f0-da5a161ef437 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.237255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.237908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.238218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.238489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.239117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.241083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.264225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10281-bdba-4f7a-976e-aa88334001df admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.268196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.268842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.269120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.269418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.278500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.279916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.290367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 932/1862] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:04.291220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.302684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a6a7-6d9b-4208-ae94-2f9a93aef534 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.308431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.309262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.309541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.309814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.310583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.312311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.332862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 933/1863] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:04.332862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.343685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83b18-4a5f-4714-9b27-c47773fb25a0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.355982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.356589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 931/1864] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:04.356999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.365963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.366376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.366376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.366376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.366376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.448669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fca86-c9f7-4b86-a7d7-c683bd6f64b4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.453639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.454298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.454571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.454839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] PATH_INFO: `/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3/users/4867e019d92741e596b44d09915f167b/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.469381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0949c709d2b14f3d9d5f6ffede8147e3, user_id=4867e019d92741e596b44d09915f167b, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.472354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28e4c1-bd56-4dd4-ac66-13787086238d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.483176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 934/1865] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:04.483176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.578428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba6847-5586-4cdd-b44d-9fbfffec1228 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.596019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.596019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.596019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.596019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.597180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.597180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.601197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 932/1866] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:04 2022] PUT /identity/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3/users/4867e019d92741e596b44d09915f167b/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:04.602824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab187-ceae-4026-a49c-bfe3c3a0e42e None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.604557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab187-ceae-4026-a49c-bfe3c3a0e42e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.605342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ab187-ceae-4026-a49c-bfe3c3a0e42e None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.607019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 933/1867] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:56:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:56:04.607984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ab187-ceae-4026-a49c-bfe3c3a0e42e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.668366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ab187-ceae-4026-a49c-bfe3c3a0e42e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.672110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.672846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.673193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.673533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.675335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:04.740611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.742114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.750991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 934/1868] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:04.751939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.765170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 935/1869] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/users => generated 299 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:04.766771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.781895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052104b1-57fd-4efc-b479-dfe0c85aea39 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.823601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a65441-f0f0-4e64-8fde-70bc57dedb04 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.828678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.842850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.842850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.842850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.842850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.842850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.852674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.853639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.853639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.853863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.854970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.856556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efb284-e433-4e59-b4fa-cce8c7430f55 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.872973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 936/1870] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:04.874408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.874708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.874985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.900090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:04.902018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:04.948209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 935/1871] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/users => generated 282 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:04.949187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.959298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e4b62-9b98-4e81-bea1-618d87045447 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:04.966150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:04.966762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:04.967064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:04.967305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:04.968120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:04.969959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:04.976046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 937/1872] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:56:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:04.976046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.984825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 936/1873] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:04.985790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:04.993461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cd156-8bdd-4ab3-9afd-aa41f58d7464 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.001817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.004807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.004807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.004807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] PATH_INFO: `/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3/users/4867e019d92741e596b44d09915f167b/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.005166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540aff-0676-4bb2-abef-7aea725fa091 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.006885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0949c709d2b14f3d9d5f6ffede8147e3, user_id=4867e019d92741e596b44d09915f167b, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.009783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.010850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.011015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.011299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.011573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.012228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.014217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.026589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 938/1874] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:04 2022] PUT /identity/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3/users/4867e019d92741e596b44d09915f167b/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:05.028233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.038001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:05.039027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 937/1875] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:05.045201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.045201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.065468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca0fb9-c741-4788-9145-ae042abdcbe0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.115870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2025fe9-deb3-4860-b7a3-189fe1d80b89 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.126679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.127691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.127992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.128274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] PATH_INFO: `/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180/users/9d07ae35978c4e408071ba54955f9a34/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.134365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.135096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.136118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.136458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.146461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.148025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07203c4562a4aaf93bd9d6ca50e5180, user_id=9d07ae35978c4e408071ba54955f9a34, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.148343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1e238-e095-4195-805d-1585aeafd082 service placement] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.150764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.159897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 938/1876] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:05.164632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.164632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.164632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.192251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] MFA Rules not processed for user `4867e019d92741e596b44d09915f167b`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:05.193717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.263918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 939/1877] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:05.264920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.275043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8e9d8-3b88-45c0-beba-97269f31b064 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.280024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.280630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.281075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.281355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.282184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.283814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.290477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 939/1878] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:05 2022] PUT /identity/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180/users/9d07ae35978c4e408071ba54955f9a34/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:05.293212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.298569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 940/1879] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:05.298569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.299906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.312127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009110-f3be-4840-8a1d-38cba262e13a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.362931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57f5b6-55b2-434e-88ed-b65f5001172f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.371248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.371248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.371248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.371248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.388331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.389085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.389979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.390339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.391726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.393607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.405523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 941/1880] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:05.406505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.418777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc9238-556e-4b6e-8568-b2114bda8206 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.424413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.435281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.435281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.435281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] PATH_INFO: `/v3/projects/61f114eb72cf4c9db994ace924c98c96/users/9a7cf8757cfa434e81b3dac1c05f66ba/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.443164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.444749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6317c-d31c-4034-b7af-47241b2f181a service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.447858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f114eb72cf4c9db994ace924c98c96, user_id=9a7cf8757cfa434e81b3dac1c05f66ba, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.450488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2035f6-8e97-402b-acd3-610ce3bb314b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.455209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 940/1881] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:05.456477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136e45-a55b-4282-8b2d-73e13955277f None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.456802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136e45-a55b-4282-8b2d-73e13955277f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.457082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89136e45-a55b-4282-8b2d-73e13955277f None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.458548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 941/1882] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:56:05.464143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89136e45-a55b-4282-8b2d-73e13955277f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.472789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89136e45-a55b-4282-8b2d-73e13955277f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.477599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.478285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.478565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.478842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] PATH_INFO: `/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180/users/9d07ae35978c4e408071ba54955f9a34/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.480950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07203c4562a4aaf93bd9d6ca50e5180, user_id=9d07ae35978c4e408071ba54955f9a34, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.483645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55eb0c-5810-4580-9ccd-17ad3aac1953 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.496780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 942/1883] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:05 2022] PUT /identity/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180/users/9d07ae35978c4e408071ba54955f9a34/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:05.498310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.498758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.499208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.534366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] MFA Rules not processed for user `14ac835d2376402f82fb9618a92f978e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:05.535620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.577198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 942/1884] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:05 2022] PUT /identity/v3/projects/61f114eb72cf4c9db994ace924c98c96/users/9a7cf8757cfa434e81b3dac1c05f66ba/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:05.587403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfacd5-6079-478f-acc2-ee5ad034f5a0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.587694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfacd5-6079-478f-acc2-ee5ad034f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.587972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfacd5-6079-478f-acc2-ee5ad034f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.620245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 943/1885] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:56:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:05.621375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.622799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.641916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2acfacd5-6079-478f-acc2-ee5ad034f5a0 None None] MFA Rules not processed for user `9d07ae35978c4e408071ba54955f9a34`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:05.643456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2acfacd5-6079-478f-acc2-ee5ad034f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.701956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24fdcd-bff8-4fb1-be0e-31f74cadac9c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.754004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 943/1886] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:05.755582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.755998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.756536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.772710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.773932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.774077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.774359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] PATH_INFO: `/v3/users/4083262ddb834617855e19aad3c1e177` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.775600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4083262ddb834617855e19aad3c1e177)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.778462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.794566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] MFA Rules not processed for user `9a7cf8757cfa434e81b3dac1c05f66ba`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:05.796218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.889090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 944/1887] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:05.890363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.936758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4083262ddb834617855e19aad3c1e177'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:05.944074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4083262ddb834617855e19aad3c1e177'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:05.945002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4083262ddb834617855e19aad3c1e177'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:05.945738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4083262ddb834617855e19aad3c1e177'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:05.963289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6f728-a391-4b9a-8451-199f7a4d8128 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:05.968579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:05.969370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:05.969824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:05.970249 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:05.970408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:05.972170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:05.985499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:05.987071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:05.997312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 945/1888] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:05.998178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:05.999501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.021655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4083262ddb834617855e19aad3c1e177'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.040817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 944/1889] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:05 2022] DELETE /identity/v3/users/4083262ddb834617855e19aad3c1e177 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:06.042140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:06.043516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.097825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616b309-0319-4f54-be8c-7e8d9f53b653 service placement] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.245744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67637c1a-5321-4652-8338-aa40bf14baf2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.248565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.251239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.255054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.255616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.263108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.263108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.263108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.263108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.263768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.263937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.264682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.359205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.360871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.372351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 946/1890] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:06.373419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 945/1891] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:06 2022] POST /identity/v3/projects => generated 363 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:06.384600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:06.413554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc848d9c-244c-4248-b803-6ce527f876e0 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.465917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:06.467704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.482895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.488787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.488787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.488787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.488787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.488787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.554969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a44a-67d3-43cf-b104-39ae70ded892 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.560424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.561296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.561826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.562181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] PATH_INFO: `/v3/projects/3b86ad2835c149b188a7448d8eed4746` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.564400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b86ad2835c149b188a7448d8eed4746)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.566886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.599210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 947/1892] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:06 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:06.601082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:06.602808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.615004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9f885-a572-4e86-9faf-2c876ad8bb03 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.619934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.620608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.620947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.621226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] PATH_INFO: `/v3/users/101c34e61a3144d8b5a7fe172bf5d5e1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.622481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] RBAC: Authorizing `identity:delete_user(user_id=101c34e61a3144d8b5a7fe172bf5d5e1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.624550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.709244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b86ad2835c149b188a7448d8eed4746', 'request_id': 'req-33c9274b-17ae-45ce-a3dd-17443b4c4180'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.716621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b86ad2835c149b188a7448d8eed4746', 'request_id': 'req-33c9274b-17ae-45ce-a3dd-17443b4c4180'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.724497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b86ad2835c149b188a7448d8eed4746', 'request_id': 'req-33c9274b-17ae-45ce-a3dd-17443b4c4180'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.851960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] The token cache is being invalidate because project 3b86ad2835c149b188a7448d8eed4746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:06.853132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.854279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.857126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 946/1893] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:06 2022] DELETE /identity/v3/projects/3b86ad2835c149b188a7448d8eed4746 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:06.861059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:06.862656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:06.901039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101c34e61a3144d8b5a7fe172bf5d5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.901914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101c34e61a3144d8b5a7fe172bf5d5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.923695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101c34e61a3144d8b5a7fe172bf5d5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:06.959422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9274b-17ae-45ce-a3dd-17443b4c4180 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:06.972905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:06.973652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:06.973940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:06.974218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:06.974892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:06.976523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:06.992035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 947/1894] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:06.994416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.009078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.026444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101c34e61a3144d8b5a7fe172bf5d5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.048952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101c34e61a3144d8b5a7fe172bf5d5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.055984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 948/1895] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:06 2022] DELETE /identity/v3/users/101c34e61a3144d8b5a7fe172bf5d5e1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:07.056552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.059720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.145376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce61dae-dca6-4841-ad6a-77ea029287f2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.175554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc23ad-c6f5-49f5-bbce-d006aad9242c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.180607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.181328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.181605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.181871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] PATH_INFO: `/v3/users/1831b40189e84a729621441451cb6a3c` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.183049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1831b40189e84a729621441451cb6a3c)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.185297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.214156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4867e019d92741e596b44d09915f167b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0949c709d2b14f3d9d5f6ffede8147e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.214682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.214962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.215238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.215911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:07.216215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.217861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.229763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 948/1896] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:07.230642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.232507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.246197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff6ca-1a87-4311-994b-18e697b07353 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.250664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.252905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.252905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.252905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] PATH_INFO: `/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8/users/428e8c2d73b9444c82a7f721828dddd4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.266778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2c6ce86ba84b41a3afb2566fa280b8, user_id=428e8c2d73b9444c82a7f721828dddd4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.269588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a75a5-e77b-435e-8b6c-8c598aaa732f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.284142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:07.381547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 949/1897] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:07 2022] PUT /identity/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8/users/428e8c2d73b9444c82a7f721828dddd4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:07.382568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.382860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.383137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.403184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:07.404564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.429399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1831b40189e84a729621441451cb6a3c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.430098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1831b40189e84a729621441451cb6a3c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.437811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1831b40189e84a729621441451cb6a3c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.469257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 950/1898] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:07.470165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.471610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.529931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1831b40189e84a729621441451cb6a3c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.542774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848b38-9724-4524-9317-d12edf9dcc49 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.544837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1831b40189e84a729621441451cb6a3c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:07.547824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4867e019d92741e596b44d09915f167b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0949c709d2b14f3d9d5f6ffede8147e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.548790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.549286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.549726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.550520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.551869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 949/1899] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:07 2022] DELETE /identity/v3/users/1831b40189e84a729621441451cb6a3c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:07.554939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.554939 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.556462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.616506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:07.639478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9064f-b151-496d-af58-1b865ad25dc0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.644458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.645178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.645459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.645733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.646384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.647986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.658282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 950/1900] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:07.659192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.661019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.679155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 951/1901] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:07 2022] POST /identity/v3/projects => generated 355 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:07.679155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.732191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03214085-3443-4e9d-b1ed-9f6786f4c12d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.749890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3e58f-597f-42dd-b200-5c16c379643f tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.756128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.756828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.757141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.760909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] PATH_INFO: `/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8/users/428e8c2d73b9444c82a7f721828dddd4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.760909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2c6ce86ba84b41a3afb2566fa280b8, user_id=428e8c2d73b9444c82a7f721828dddd4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.762726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.774049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 952/1902] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:07 2022] PUT /identity/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8/users/428e8c2d73b9444c82a7f721828dddd4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:07.777719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.782974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.783812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.784168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.784527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.794764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.796290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2819b36-aa7c-48b9-9d5d-cb878ccb3747 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.805780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 951/1903] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:07.807573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.807860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.808152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.840491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] MFA Rules not processed for user `428e8c2d73b9444c82a7f721828dddd4`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:07.841996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.857178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f626-6872-471d-b8c4-b54dbef4e2ec admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.864917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4867e019d92741e596b44d09915f167b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0949c709d2b14f3d9d5f6ffede8147e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.868737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.868737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.868737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.873405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:07.874694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.877816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.896033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 953/1904] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:07.897068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.899916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:07.944789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 952/1905] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:07.950436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:07.961563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e5a17-a747-4acb-9906-34a36d391f3f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.967218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4867e019d92741e596b44d09915f167b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0949c709d2b14f3d9d5f6ffede8147e3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.968007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.968392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.968944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] PATH_INFO: `/v3/projects/06723e6163c348318ae4da617d8a1d4e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.970196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06723e6163c348318ae4da617d8a1d4e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.972267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:07.975775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d7240-9fae-4fd4-9326-00d465a81e84 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:07.981275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:07.981905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:07.982183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:07.982502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] PATH_INFO: `/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:07.984020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=752bd72aaa0f47b6a4c0b611e9ba59e1)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:07.986313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:09.001169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06723e6163c348318ae4da617d8a1d4e', 'request_id': 'req-b9674fde-8beb-4c44-8686-6100601acc07'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.002919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752bd72aaa0f47b6a4c0b611e9ba59e1', 'request_id': 'req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.008872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752bd72aaa0f47b6a4c0b611e9ba59e1', 'request_id': 'req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.015733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752bd72aaa0f47b6a4c0b611e9ba59e1', 'request_id': 'req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.103628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06723e6163c348318ae4da617d8a1d4e', 'request_id': 'req-b9674fde-8beb-4c44-8686-6100601acc07'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.110636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06723e6163c348318ae4da617d8a1d4e', 'request_id': 'req-b9674fde-8beb-4c44-8686-6100601acc07'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.122085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] The token cache is being invalidate because project 06723e6163c348318ae4da617d8a1d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:09.122414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.123162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.124780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 953/1906] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:07 2022] DELETE /identity/v3/projects/06723e6163c348318ae4da617d8a1d4e => generated 0 bytes in 1175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:09.125825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:09.127427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.195180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] The token cache is being invalidate because project 752bd72aaa0f47b6a4c0b611e9ba59e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:09.195475 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.196129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.198560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 954/1907] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:07 2022] DELETE /identity/v3/projects/752bd72aaa0f47b6a4c0b611e9ba59e1 => generated 0 bytes in 1302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:09.199211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:09.200565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.206653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9674fde-8beb-4c44-8686-6100601acc07 tempest-VolumeQuotaClassesTest-1180707375 tempest-VolumeQuotaClassesTest-1180707375-project-admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:09.215956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:09.215956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:09.215956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:09.215956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:09.216345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.274061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9193f02-4856-41ce-a1dc-5bccc83ccb4f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:09.280876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:09.282316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:09.292696 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 954/1908] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:09.293751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:09.294937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.339628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:09.340383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:09.340773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:09.341127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] PATH_INFO: `/v3/users/57db7a0affef431c8d1ec206cf2b28d0` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:09.342342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57db7a0affef431c8d1ec206cf2b28d0)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:09.344392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:09.376263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579ac2d-e176-4e13-ad09-851f833577bb service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:09.381651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:09.382356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:09.382624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:09.382905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] PATH_INFO: `/v3/projects/83fc0ff07e764a789e40530a29b56af0` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:09.384265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fc0ff07e764a789e40530a29b56af0)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:09.388786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:09.588749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57db7a0affef431c8d1ec206cf2b28d0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.589236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fc0ff07e764a789e40530a29b56af0', 'request_id': 'req-d820490c-9f41-403c-a15d-712aad54e585'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.596908 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57db7a0affef431c8d1ec206cf2b28d0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.597849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57db7a0affef431c8d1ec206cf2b28d0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.598716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57db7a0affef431c8d1ec206cf2b28d0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.747272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fc0ff07e764a789e40530a29b56af0', 'request_id': 'req-d820490c-9f41-403c-a15d-712aad54e585'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.754625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fc0ff07e764a789e40530a29b56af0', 'request_id': 'req-d820490c-9f41-403c-a15d-712aad54e585'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.766836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] The token cache is being invalidate because project 83fc0ff07e764a789e40530a29b56af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:09.767254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.767993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d820490c-9f41-403c-a15d-712aad54e585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.772228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 955/1909] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:09 2022] DELETE /identity/v3/projects/83fc0ff07e764a789e40530a29b56af0 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:09.772228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119eb722-3ec9-4cd3-9537-dec15b088ede None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:09.772228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119eb722-3ec9-4cd3-9537-dec15b088ede None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:09.772228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119eb722-3ec9-4cd3-9537-dec15b088ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:09.803185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-119eb722-3ec9-4cd3-9537-dec15b088ede None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:09.804701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-119eb722-3ec9-4cd3-9537-dec15b088ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.861007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57db7a0affef431c8d1ec206cf2b28d0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 956/1910] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 957/1911] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:56:09.891960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:09.894130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 955/1912] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:09 2022] DELETE /identity/v3/users/57db7a0affef431c8d1ec206cf2b28d0 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:09.895117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:09.896674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:09.901569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.006052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaca940-b197-431f-8aa9-dd3cec37bd26 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.012515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.013503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.014058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.014493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.016824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.018545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34caa5c-501f-40f1-82c4-340328e52211 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.024443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.028203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.035474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.035474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] PATH_INFO: `/v3/users/9d07ae35978c4e408071ba54955f9a34` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.035474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d07ae35978c4e408071ba54955f9a34)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.035474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.088351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.089544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3eded5-171d-4f64-99c1-3a0f9817c034 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.102717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 958/1913] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:10.104328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf964c-7248-49a5-8506-4edb1fd3d2bc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.104619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf964c-7248-49a5-8506-4edb1fd3d2bc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.104943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf964c-7248-49a5-8506-4edb1fd3d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.132948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e9bf964c-7248-49a5-8506-4edb1fd3d2bc None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:10.135104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e9bf964c-7248-49a5-8506-4edb1fd3d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.148021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 959/1914] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:10.148021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afba20-1819-4b0c-b35a-b4ba8b792cea None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.148021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afba20-1819-4b0c-b35a-b4ba8b792cea None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.148021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afba20-1819-4b0c-b35a-b4ba8b792cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.168694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-75afba20-1819-4b0c-b35a-b4ba8b792cea None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:10.170367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-75afba20-1819-4b0c-b35a-b4ba8b792cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.178653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 960/1915] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:10.179883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.180164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.180438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.204382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:10.205674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.216577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d07ae35978c4e408071ba54955f9a34'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.225755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d07ae35978c4e408071ba54955f9a34'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.227131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d07ae35978c4e408071ba54955f9a34'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.227995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d07ae35978c4e408071ba54955f9a34'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.307700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 961/1916] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:10.308732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.310067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.317358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d07ae35978c4e408071ba54955f9a34'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.329997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96ad9-9429-40ac-8c01-b4b65f74ed46 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.335090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.335774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.337560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.337560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] PATH_INFO: `/v3/users/9b88b473ff024e088e009c7b1a9a1ada` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.337560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b88b473ff024e088e009c7b1a9a1ada)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.339423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.343368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 956/1917] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:09 2022] DELETE /identity/v3/users/9d07ae35978c4e408071ba54955f9a34 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:10.343368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.343368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.421255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef734c0f-53a4-49f2-bdbe-2ba2f5c52898 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.426774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.427901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.428254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.428640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.429464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:10.429949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.431636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.446968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 957/1918] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:10.448364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.451020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.512632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd28df-89e9-4c04-87e9-5a197729bafd admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.517806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.518434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.518715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.521194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.521194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.535071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.536546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.555947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 958/1919] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:10.555947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.555947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.613575 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:10.629227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b88b473ff024e088e009c7b1a9a1ada'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.630157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b88b473ff024e088e009c7b1a9a1ada'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.638901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b88b473ff024e088e009c7b1a9a1ada'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.646073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abf8b7-130c-4070-8ece-8b5a6f4c5311 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.653676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.654262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.654630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.655004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.659522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.674297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.676110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.688421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 959/1920] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:10.689113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.690685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.711123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde6e02-080f-4307-8f8b-ce66f5b74a77 service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.717780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.718559 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.718959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.719467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.720237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.722324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:10.764543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b88b473ff024e088e009c7b1a9a1ada'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.782283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-691b356b-ea01-40ac-a9aa-6040bc7ab64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b88b473ff024e088e009c7b1a9a1ada'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:10.794000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 962/1921] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:10 2022] DELETE /identity/v3/users/9b88b473ff024e088e009c7b1a9a1ada => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:10.795820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.796110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.796453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.821890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:10.823816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.865167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 960/1922] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/projects => generated 363 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:10.866369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.921445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:10.935218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 963/1923] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:10.948784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:10.953709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:10.981641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1423594-d9b0-4d21-9f5b-14a76a7e2ed5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:10.989344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:10.990248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:10.990674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:10.991112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:10.991909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:10.994170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.048686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ad9fc-5ccf-416a-9741-ff008cbfe9b2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.120765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 961/1924] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:10 2022] POST /identity/v3/users => generated 282 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:11.122093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.125141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.128917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.130850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.131339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.131894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.141096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81028d-85a3-4a8d-b06d-580a0bef08ab admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.146566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.146996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.147230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.147509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.148337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] PATH_INFO: `/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.149551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07203c4562a4aaf93bd9d6ca50e5180)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.150335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.152693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.180431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 964/1925] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:11.180431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.193273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bfabd-5ea9-46d1-93fe-b302879d3016 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.202930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.202930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.202930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.202930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.205342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.206609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.221728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 965/1926] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:11.226529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.237770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fd402-2624-4e7a-9831-053fec99b040 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.243093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.247627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.247627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.247627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.247627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.247627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.268971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 966/1927] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:11.271635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.286277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07203c4562a4aaf93bd9d6ca50e5180', 'request_id': 'req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.290212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1540e5-9bfc-48cb-91a5-8e5b0d5a2add admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.293118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07203c4562a4aaf93bd9d6ca50e5180', 'request_id': 'req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.305395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07203c4562a4aaf93bd9d6ca50e5180', 'request_id': 'req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.356290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.357049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.357331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.357613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] PATH_INFO: `/v3/projects/428e204326024f038328718ccd256144/users/b71d345fb1d14786b5bff255c017cf2f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.379820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428e204326024f038328718ccd256144, user_id=b71d345fb1d14786b5bff255c017cf2f, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.379820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d039044-d5a2-429f-ae0c-f6e0a53a0fa1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.411548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] The token cache is being invalidate because project a07203c4562a4aaf93bd9d6ca50e5180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:11.414255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.414255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.415590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 962/1928] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:11 2022] DELETE /identity/v3/projects/a07203c4562a4aaf93bd9d6ca50e5180 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:11.416467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.420535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.438684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f31a-3d59-419d-b04c-6d2769b9a78c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.444895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.448640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.448640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.448640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] PATH_INFO: `/v3/projects/b91ca21043ba41afa802db4d2523b774` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.448640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b91ca21043ba41afa802db4d2523b774)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.450943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.523320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 967/1929] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:11 2022] PUT /identity/v3/projects/428e204326024f038328718ccd256144/users/b71d345fb1d14786b5bff255c017cf2f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:11.523320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1acb-28d9-401a-ab4a-525f3a709b9d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.523320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1acb-28d9-401a-ab4a-525f3a709b9d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.523320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d1acb-28d9-401a-ab4a-525f3a709b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.541975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ac7d1acb-28d9-401a-ab4a-525f3a709b9d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:11.543541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ac7d1acb-28d9-401a-ab4a-525f3a709b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.633636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 968/1930] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:11.633636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db76c7-d2df-4534-88c6-d40dbad0e31a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.633636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db76c7-d2df-4534-88c6-d40dbad0e31a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.633636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db76c7-d2df-4534-88c6-d40dbad0e31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.665581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91ca21043ba41afa802db4d2523b774', 'request_id': 'req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.672128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91ca21043ba41afa802db4d2523b774', 'request_id': 'req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.681127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91ca21043ba41afa802db4d2523b774', 'request_id': 'req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.683201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e2db76c7-d2df-4534-88c6-d40dbad0e31a None None] MFA Rules not processed for user `b71d345fb1d14786b5bff255c017cf2f`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:11.693018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e2db76c7-d2df-4534-88c6-d40dbad0e31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.774946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 969/1931] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:11.776294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.776582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.776950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.791648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] The token cache is being invalidate because project b91ca21043ba41afa802db4d2523b774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:11.791957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.792795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:11.794728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 963/1932] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:11 2022] DELETE /identity/v3/projects/b91ca21043ba41afa802db4d2523b774 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:11.795793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.797463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.804255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:11.805900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.867429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca227d-c7cf-4fc1-944f-0c62b1ae83ca admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.872631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.873264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.873541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.873807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.874423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.874555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 970/1933] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:11.879440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:11.879607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:11.879758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:11.889995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c207595-c02d-4a77-9a52-f1f0e3557aa0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:11.895624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:11.896501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:11.896726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:11.897005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] PATH_INFO: `/v3/users/963d819c41794194bf063fce8f25d99a` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:11.898226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=963d819c41794194bf063fce8f25d99a)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:11.900225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.013025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 964/1934] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:11 2022] POST /identity/v3/projects => generated 363 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:12.013940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.082748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26073-fd23-445a-9432-a9b7c1a2b32b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.089672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.090295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.090576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.090850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.091574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:12.091881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.093672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.110885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 965/1935] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:12.110885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.112060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963d819c41794194bf063fce8f25d99a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.113455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.115445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963d819c41794194bf063fce8f25d99a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.130972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963d819c41794194bf063fce8f25d99a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.138384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dadbdd-d0ef-4fb7-85d1-e92d276b04e8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.143393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.144162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.144552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.144861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] PATH_INFO: `/v3/projects/02fc2813b8544abb8d31896f84e331f5` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.146169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fc2813b8544abb8d31896f84e331f5)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.148091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.255552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963d819c41794194bf063fce8f25d99a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.273886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963d819c41794194bf063fce8f25d99a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.285043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 971/1936] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:11 2022] DELETE /identity/v3/users/963d819c41794194bf063fce8f25d99a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:12.285860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.288818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.293404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fc2813b8544abb8d31896f84e331f5', 'request_id': 'req-772fca3e-efbc-4e0b-bd51-e3b5015baf02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.302189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fc2813b8544abb8d31896f84e331f5', 'request_id': 'req-772fca3e-efbc-4e0b-bd51-e3b5015baf02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.319331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fc2813b8544abb8d31896f84e331f5', 'request_id': 'req-772fca3e-efbc-4e0b-bd51-e3b5015baf02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.364047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b22b3-56d8-4376-a3f4-a0db8e08cf3e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.370558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.370781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.371241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.371363 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.371937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.373943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.415542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] The token cache is being invalidate because project 02fc2813b8544abb8d31896f84e331f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:12.416700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.416700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.418412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 966/1937] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:12 2022] DELETE /identity/v3/projects/02fc2813b8544abb8d31896f84e331f5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:12.419213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.422385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.504966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fca3e-efbc-4e0b-bd51-e3b5015baf02 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.508563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 972/1938] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:12 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:12.508563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.510201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.510971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.511253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.511529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.511960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.512171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.514178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.539023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a3d55-e6a4-47b7-9705-5b702a9ad8c2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.546161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.547304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.547729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.548272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] PATH_INFO: `/v3/users/4867e019d92741e596b44d09915f167b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.549833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4867e019d92741e596b44d09915f167b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.552020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.664639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 967/1939] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:12 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:12.665725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.670589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.747675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46588-018b-42ad-8585-f7b37b6612f3 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.753811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.753811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.753811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.753811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.754445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.756190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.770984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 968/1940] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:12.771873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.783363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfdeff-fcee-4deb-a339-cb41abce8944 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.788574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.789230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.790025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.790025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.790370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.792826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:12.846632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4867e019d92741e596b44d09915f167b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.849083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4867e019d92741e596b44d09915f167b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.858116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4867e019d92741e596b44d09915f167b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:12.916846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 969/1941] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:12 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:12.916846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:12.917945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:12.940121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837f607-34f9-4148-b847-976937fb70f1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:12.952258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:12.952928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:12.953269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:12.953681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] PATH_INFO: `/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71/users/74a30bc7aa88415c86fb374258e23cd1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:12.972880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0c5b7a3e644101ab86cabd806cbd71, user_id=74a30bc7aa88415c86fb374258e23cd1, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:12.975950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.043958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4867e019d92741e596b44d09915f167b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.056480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4867e019d92741e596b44d09915f167b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.062326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 973/1942] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:12 2022] DELETE /identity/v3/users/4867e019d92741e596b44d09915f167b => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:13.063164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.064493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.119139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 970/1943] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:12 2022] PUT /identity/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71/users/74a30bc7aa88415c86fb374258e23cd1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:13.119139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.121215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.147618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:13.156891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aef1a-5db0-47f8-a8ad-b3a4e39264fa admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.250555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.251884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.253839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.254294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] PATH_INFO: `/v3/projects/9071337458c84bed954e6da88d460a31` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.258018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9071337458c84bed954e6da88d460a31)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.263088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.295483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8344f2-21f6-43c8-95d6-9a1bcc0dcb36 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.306646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.307540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.307987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.308418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.323432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.325374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.348335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 971/1944] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:13.349363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.377471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff8cc5-c5b4-43b4-8dc1-31159c2c95ac admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.396484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.397693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.398568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.398936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.399700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.402344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.418677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 972/1945] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:13.422893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.433182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f228c-b8fe-410f-b57d-827294ef41f7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.438770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.439389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.439669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.439941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.440593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.442425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.456697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 973/1946] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:13.456697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.469242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b400a5-9d41-4256-b44f-08e0165e5446 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.470058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9071337458c84bed954e6da88d460a31', 'request_id': 'req-c49cf259-fe52-46ca-8013-2328da848437'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.540151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:13.601712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.602418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.602764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9071337458c84bed954e6da88d460a31', 'request_id': 'req-c49cf259-fe52-46ca-8013-2328da848437'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.602885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.603050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] PATH_INFO: `/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71/users/74a30bc7aa88415c86fb374258e23cd1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.612434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0c5b7a3e644101ab86cabd806cbd71, user_id=74a30bc7aa88415c86fb374258e23cd1, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.613112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1803f-82bd-44d4-9127-6fb0a508ef4c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.614668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9071337458c84bed954e6da88d460a31', 'request_id': 'req-c49cf259-fe52-46ca-8013-2328da848437'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.629882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] The token cache is being invalidate because project 9071337458c84bed954e6da88d460a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:13.630237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.631038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:13.636120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 974/1948] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:13 2022] DELETE /identity/v3/projects/9071337458c84bed954e6da88d460a31 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:13.636120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.636388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 974/1948] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:13 2022] PUT /identity/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71/users/74a30bc7aa88415c86fb374258e23cd1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:13.636492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.636928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3324d0-ca45-4def-972e-e4c0df945088 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.637284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3324d0-ca45-4def-972e-e4c0df945088 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.637632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3324d0-ca45-4def-972e-e4c0df945088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.653436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49cf259-fe52-46ca-8013-2328da848437 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.679146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5c3324d0-ca45-4def-972e-e4c0df945088 None None] MFA Rules not processed for user `74a30bc7aa88415c86fb374258e23cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:13.680750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5c3324d0-ca45-4def-972e-e4c0df945088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.729633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.730404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.730678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.730964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] PATH_INFO: `/v3/projects/b0f772bda05d4a74ae4d83057c08726f/users/1c055fe5ccb24057bcf5ac4fe89b7a67/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.746340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f772bda05d4a74ae4d83057c08726f, user_id=1c055fe5ccb24057bcf5ac4fe89b7a67, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.749988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:13.768183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 975/1949] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:13.769487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.769771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.770047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.797645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:13.798897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.808011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 976/1950] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:13.808902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.814403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.883946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b2c68-dfb5-4b05-b0a8-39b70cf6f022 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.887505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 975/1951] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:13 2022] PUT /identity/v3/projects/b0f772bda05d4a74ae4d83057c08726f/users/1c055fe5ccb24057bcf5ac4fe89b7a67/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:13.897131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:13.898785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:13.934422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.935694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.936083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.936463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.965022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc766a-057a-4934-927b-ffeffe3ad8fc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:13.972168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:13.973211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:13.973663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:13.974121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] PATH_INFO: `/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:13.975853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0949c709d2b14f3d9d5f6ffede8147e3)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:13.979629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.013455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.014819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c72659-0e9e-462c-8d5b-bbab8fe25a14 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.027608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 977/1952] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:14.028945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db16-37d0-4af5-aacc-5f6a61d2cf65 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.029194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db16-37d0-4af5-aacc-5f6a61d2cf65 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.029484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58db16-37d0-4af5-aacc-5f6a61d2cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.055220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7d58db16-37d0-4af5-aacc-5f6a61d2cf65 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:14.058382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d58db16-37d0-4af5-aacc-5f6a61d2cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.065890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 978/1953] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:14.067311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.067804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.068115 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.098328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] MFA Rules not processed for user `1c055fe5ccb24057bcf5ac4fe89b7a67`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:14.099604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.131145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0949c709d2b14f3d9d5f6ffede8147e3', 'request_id': 'req-7247f66e-7320-4d20-a0cb-2a735060f784'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:14.179288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 979/1954] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:14.179288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.188869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f815e7a-6557-4f87-a1aa-ae743ca138bf None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.232402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0949c709d2b14f3d9d5f6ffede8147e3', 'request_id': 'req-7247f66e-7320-4d20-a0cb-2a735060f784'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:14.239354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0949c709d2b14f3d9d5f6ffede8147e3', 'request_id': 'req-7247f66e-7320-4d20-a0cb-2a735060f784'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:14.246602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.247338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.247801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.248266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.249116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:14.249514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.251216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6223-5380-478c-8402-aa8d31e2c396 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.254886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] The token cache is being invalidate because project 0949c709d2b14f3d9d5f6ffede8147e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:14.255179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:14.255864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:14.257425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 976/1955] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:13 2022] DELETE /identity/v3/projects/0949c709d2b14f3d9d5f6ffede8147e3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:14.258179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.259603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.266341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 980/1956] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:14.267535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.267822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.268095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.273753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247f66e-7320-4d20-a0cb-2a735060f784 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.278916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.279528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.279812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.280080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.280739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.282655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ad40-8796-4302-8ad6-4edf8b398bd8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.289711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:14.290935 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.304260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 981/1957] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:14.304260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.305457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.317599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366588ad-b8f8-4e06-855b-c43f66da718c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.322807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.323416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.323717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.324120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.324790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.326676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.424858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 977/1958] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:14.424858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.424858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.424858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.454121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:14.454121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.454542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 982/1959] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:14.455721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.462198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.520785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 978/1960] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:14.520785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.532805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fadf0-413d-4d04-9932-6ff261a431b2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.536231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418e19d-4cce-4022-b81e-f509a616ff35 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.537985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.538602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.538951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.539250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.539894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.540995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.541718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.541876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.542002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.542272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.544145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:14.609674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.611097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.620602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 983/1961] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:14.621538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.631776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26f6ca-8a77-4c46-92eb-f160afdabd3d service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.637251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.637855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.638170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.638400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.641790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.641790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.667619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 979/1962] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:14.670089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.682643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b96-6bf6-46b2-a9e3-79bcfa62b60e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.687381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.688035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.688424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.688585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.689270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:14.689581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.691180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.709722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 980/1963] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:14.709722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.722463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a9cd5-72bf-4f69-8fca-b3b1b9daa263 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.728100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.729276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.729556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.730200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.730618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.732456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.745376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 981/1964] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:14.745376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.756611 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e294a-8ee8-4a97-9792-b9bd51535bab admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.768767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.773824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.773824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.773824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.773824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.773824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.802598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 984/1965] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:14.802598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.814565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0fd66-d380-414f-891e-f85031558d52 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.819968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:14.820687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:14.820972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:14.821246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] PATH_INFO: `/v3/projects/4e82e56d71674a038b8dd22191bcb5d7/users/fd4fb85c3fcd4317ad287d5f01cb87b1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:14.839236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e82e56d71674a038b8dd22191bcb5d7, user_id=fd4fb85c3fcd4317ad287d5f01cb87b1, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:14.842157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:14.859041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 982/1966] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:14.863068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:14.882633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:14.941571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 985/1967] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:14 2022] PUT /identity/v3/projects/4e82e56d71674a038b8dd22191bcb5d7/users/fd4fb85c3fcd4317ad287d5f01cb87b1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:14.941936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29be78-57d2-463d-874f-3e77492dcc4b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:14.942048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.029306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.029842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.030117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.030385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.033007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.034262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.038347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953117f-3a38-4ebd-95b1-9be8239bc179 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.049810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 983/1968] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:15.053355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.053677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.054002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.054681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.054681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.055149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.058873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed9979-18bc-4a20-8dc9-ba5c5856e5d3 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.067277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d7734-5f14-4aba-b548-cb63a363f4fc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.074181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.075026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.075342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.075659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.076275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.078046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.088760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 984/1969] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:15.092155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.099806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083e4cb-58d2-43b2-986e-fa783ce44e3c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.105119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.105718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.106005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.106279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.106927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.108537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.120175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 985/1970] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:15.121031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.131322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90638353-1f99-4029-a4bd-bb66c43153de admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.136283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.138937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.138937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.138937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] PATH_INFO: `/v3/projects/4e82e56d71674a038b8dd22191bcb5d7/users/fd4fb85c3fcd4317ad287d5f01cb87b1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.139489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e82e56d71674a038b8dd22191bcb5d7, user_id=fd4fb85c3fcd4317ad287d5f01cb87b1, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.142214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.153256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 986/1971] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:15 2022] PUT /identity/v3/projects/4e82e56d71674a038b8dd22191bcb5d7/users/fd4fb85c3fcd4317ad287d5f01cb87b1/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:15.154024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.182703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 986/1972] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:14 2022] POST /identity/v3/users => generated 303 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:15.184936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.185749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.186054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.220584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] MFA Rules not processed for user `fd4fb85c3fcd4317ad287d5f01cb87b1`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:15.221858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:15.227963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba276fe-801f-4b95-967a-293ebb6dea97 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.235945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.235945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.235945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.235945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] PATH_INFO: `/v3/projects/8359395fb8fc4967a446471664d6f4f0/users/77eb077134f24e32affb3208f16ce583/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.250856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8359395fb8fc4967a446471664d6f4f0, user_id=77eb077134f24e32affb3208f16ce583, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.253796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.307289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 987/1973] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:15.308182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.322403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cba54b-de9c-4d6d-b4be-684fb241cdc5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.329402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.330089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.330573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.331345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.332251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.334841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.347763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 988/1974] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:15.348134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.371497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 987/1975] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:15 2022] PUT /identity/v3/projects/8359395fb8fc4967a446471664d6f4f0/users/77eb077134f24e32affb3208f16ce583/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:15.373100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.450141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f201d60-3d66-4ccd-937c-351779d7e7be admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.461378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e81fb-df91-457b-9fb3-81f0749adc41 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.466756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.467376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.467651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.467918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.468605 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.470199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a7223-b7bd-4f9b-9c58-f314ef5dcd9a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.481137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 988/1976] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:15.482506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.482795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.483067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.511259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.511741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.512031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.516852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.516968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] MFA Rules not processed for user `77eb077134f24e32affb3208f16ce583`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:15.516968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:15.579395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.579395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.596581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 989/1977] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:15.596700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 989/1978] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:15.596782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.600044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.603352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b9e1f-974c-43a0-b2ff-f787b0e9d969 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.608228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.608928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.609192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.609464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] PATH_INFO: `/v3/projects/ffda9549a4db451a831cbb21923d5e38/users/86030a2f71844c9e9ab894ef3955e5c0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.624932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffda9549a4db451a831cbb21923d5e38, user_id=86030a2f71844c9e9ab894ef3955e5c0, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.627271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc6697-5645-4b59-a74c-7a95e7c566f7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.627577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb480dc-2ad7-4e27-a2eb-ed28e66ebe1b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.643358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.643994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.644274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.644549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.645217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.647161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.756966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 990/1979] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:15 2022] PUT /identity/v3/projects/ffda9549a4db451a831cbb21923d5e38/users/86030a2f71844c9e9ab894ef3955e5c0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:15.767839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.768119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.768394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.801118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 990/1980] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:15.802292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] MFA Rules not processed for user `86030a2f71844c9e9ab894ef3955e5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:15.806561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.806674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:15.859960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5594c4b-40ce-4ea5-bb66-350d9d038d09 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.863708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.864320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.864602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.864966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.866777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:15.876392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 991/1981] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:15.877412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.928760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.930200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.940434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 991/1982] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:15.941279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:15.945765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5179ffd-95c8-49ba-8941-0c4745e416d4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.951173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.951783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.952052 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.952409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.953083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.953207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce8317-c3c1-4c5e-85d1-83afeea3a6cd service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:15.955044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:15.958569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:15.959317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:15.959726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:15.960001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:15.960625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:15.963023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.055628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 992/1983] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:16.059983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.067499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:16.071448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20cc0-41a0-4a07-ae50-ed9397aa032e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.076678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 992/1984] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:15 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:16.076875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.077492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.077767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.078039 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.078687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.080279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.084940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.091711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 993/1985] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:16.102790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.115355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090327f-5fd3-4a57-b025-927b85875f1b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.184305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e97c4-b899-4af2-8814-9d285aa0ebd7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.190182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.191140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.191427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.192623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.193295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.195314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.195866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.196491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.196851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.197069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.198288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.205632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.217021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 994/1986] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:16.221310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.233634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6e5ea-c7cb-4ca2-9e53-bbedfe16ad9a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.238408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.239129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.239476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.239814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] PATH_INFO: `/v3/projects/db0352401e7e414fba9ccef65acf8e4b/users/9a94e4fef3d94bb5bf6ecda16dba30fd/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.256059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0352401e7e414fba9ccef65acf8e4b, user_id=9a94e4fef3d94bb5bf6ecda16dba30fd, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.258984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda4360-0031-4abe-8f21-92440f7ad332 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.335429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 993/1987] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:16 2022] POST /identity/v3/users => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:16.339273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.354288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a942ac2-a9a0-4ea1-ab23-2ace89b80220 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.354375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 995/1988] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:16 2022] PUT /identity/v3/projects/db0352401e7e414fba9ccef65acf8e4b/users/9a94e4fef3d94bb5bf6ecda16dba30fd/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:16.363260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.363556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.363833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.398036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] MFA Rules not processed for user `9a94e4fef3d94bb5bf6ecda16dba30fd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:16.399852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:16.432489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.433275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.433666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.434020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.434799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.437867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.449714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 994/1989] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:16.453531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.463955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a54f57-c9a4-4f03-a2a2-98649e1d95cd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.469288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.469941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.470213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.470481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] PATH_INFO: `/v3/projects/880d76edb3b54339925807870ed5ecee/users/c5f729e00a8c431b80015b9a22349ae4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.486325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880d76edb3b54339925807870ed5ecee, user_id=c5f729e00a8c431b80015b9a22349ae4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.489238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.490038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 996/1990] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:16.495173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.505755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e531b-b2ef-4c37-bdef-652ca0719a13 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.512467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.513109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.513394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.518629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.518629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.518629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.573921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 995/1991] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:16 2022] PUT /identity/v3/projects/880d76edb3b54339925807870ed5ecee/users/c5f729e00a8c431b80015b9a22349ae4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:16.574852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.635930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04b7b5-c9b2-4a4c-8851-caceb6aa6e63 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.641218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 997/1992] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:16 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:16.641218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.702156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.702793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.703189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.703471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] PATH_INFO: `/v3/users/1c055fe5ccb24057bcf5ac4fe89b7a67` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.704626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c055fe5ccb24057bcf5ac4fe89b7a67)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.708361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.724945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6d6ae-c7c8-4ced-9e2d-23304c0b0995 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.730561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.731181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.731458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.731915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.733030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.734793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.745058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 998/1993] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:16.745844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.755144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3c7a2-3185-4ca8-a418-46617e366d15 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.759841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.760417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.760720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.760995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.761601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.763464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.847840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 999/1994] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:16 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:16.847840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.856684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c0ec9-26a6-43b6-8617-33156648c414 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.861803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.862479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.862815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.863100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] PATH_INFO: `/v3/projects/880d76edb3b54339925807870ed5ecee/users/c5f729e00a8c431b80015b9a22349ae4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.867408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880d76edb3b54339925807870ed5ecee, user_id=c5f729e00a8c431b80015b9a22349ae4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.870296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.883601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1000/1995] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:16 2022] PUT /identity/v3/projects/880d76edb3b54339925807870ed5ecee/users/c5f729e00a8c431b80015b9a22349ae4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:16.884455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:16.932147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c055fe5ccb24057bcf5ac4fe89b7a67'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:16.933026 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c055fe5ccb24057bcf5ac4fe89b7a67'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:16.941369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c055fe5ccb24057bcf5ac4fe89b7a67'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:16.952694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e20832-d763-4211-842d-e2ec1aa3f1e5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:16.957770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:16.958432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.958715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.961412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:16.961412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:16.961412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35ff47-99ff-42e9-8f7e-e8b5795eedbb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:16.972986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1001/1996] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:16.973712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8564-3945-46f2-ba36-e07e24b5d81a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:16.973994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8564-3945-46f2-ba36-e07e24b5d81a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:16.974267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a8564-3945-46f2-ba36-e07e24b5d81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.006897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7e7a8564-3945-46f2-ba36-e07e24b5d81a None None] MFA Rules not processed for user `c5f729e00a8c431b80015b9a22349ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:17.008121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7e7a8564-3945-46f2-ba36-e07e24b5d81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.010397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c055fe5ccb24057bcf5ac4fe89b7a67'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:17.026963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c055fe5ccb24057bcf5ac4fe89b7a67'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:17.038632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 996/1997] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:16 2022] DELETE /identity/v3/users/1c055fe5ccb24057bcf5ac4fe89b7a67 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:17.039453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.041938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.089364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1002/1998] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:16 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:17.090639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.090746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:17.090928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.091202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.112925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:17.114325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.128389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18efd8-8f7e-4635-9a4f-e5912dbc325b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.130409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1003/1999] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:17.131413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.132870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.135902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.136778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.137061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.137337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] PATH_INFO: `/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1/users/ac2848a6d33d40d4bc7f16514ddce3d3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.155927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1bbc0ffdae4b6abb217be9b517e5a1, user_id=ac2848a6d33d40d4bc7f16514ddce3d3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.158816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.191585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d955aa-bf24-4293-b1df-63799cd7247a None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.195197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.195809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.196081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.196346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.238281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 997/2000] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:17 2022] PUT /identity/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1/users/ac2848a6d33d40d4bc7f16514ddce3d3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:17.239294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.241107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.261026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.262452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.272320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1004/2001] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:17.274970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.286446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b6a6f-a918-41a8-aee7-05b255831ed7 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.309175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111311e-7369-43fa-be77-c8c06601541c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.313349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.314034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.314385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.314745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.318628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:17.365450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.366135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.366369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.366636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.367317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.369673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.381636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1005/2002] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:17.387206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.398521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02ca42-8b50-4b29-8914-3be0fd628f26 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.403757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.404369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.404704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.405122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] PATH_INFO: `/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1/users/ac2848a6d33d40d4bc7f16514ddce3d3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.407381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1bbc0ffdae4b6abb217be9b517e5a1, user_id=ac2848a6d33d40d4bc7f16514ddce3d3, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.416725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.418772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8993bf1-820d-4900-b92e-05e2bb20f619 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.423308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.448107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 998/2003] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:17.448287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1006/2004] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:17 2022] PUT /identity/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1/users/ac2848a6d33d40d4bc7f16514ddce3d3/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:17.458496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.458784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.459182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.492397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] MFA Rules not processed for user `ac2848a6d33d40d4bc7f16514ddce3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:17.495119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.570055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 999/2005] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:17.582467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.641536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb036-ec56-4ae6-acca-60daf4162da8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.645218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.645825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.646103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.646378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.655870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.657334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.668091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1007/2006] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:17.820108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.821826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.834002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524d06c-a35a-4b19-91da-90b2a7918c89 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.837974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.838572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.838849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.839124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.840994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.896536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.905510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfb1da-e4b2-4334-beae-0c8267840807 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:17.907947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.909749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:17.909884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:17.910275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:17.910401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:17.911449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:17.912598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.929931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1000/2007] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:17.981139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:17.981139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:17.993497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:17.995585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:18.043852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1008/2008] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:18.110833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74c827-e987-425f-aa6f-15bb5c74e98f service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:18.115924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:18.116574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:18.116884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:18.117157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] PATH_INFO: `/v3/projects/b0f772bda05d4a74ae4d83057c08726f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:18.118526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f772bda05d4a74ae4d83057c08726f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:18.120985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:18.280636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f772bda05d4a74ae4d83057c08726f', 'request_id': 'req-f6cc7660-22d0-48c0-aaa3-469c87056aec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.364176 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f772bda05d4a74ae4d83057c08726f', 'request_id': 'req-f6cc7660-22d0-48c0-aaa3-469c87056aec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.371253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f772bda05d4a74ae4d83057c08726f', 'request_id': 'req-f6cc7660-22d0-48c0-aaa3-469c87056aec'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.383543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] The token cache is being invalidate because project b0f772bda05d4a74ae4d83057c08726f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:18.383840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.384525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.386129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1001/2009] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:17 2022] DELETE /identity/v3/projects/b0f772bda05d4a74ae4d83057c08726f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:18.390043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:18.401327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d7993-5fe3-4e27-93e4-c45ecaa2c14a service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:18.458460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:18.459135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:18.459451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:18.459729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] PATH_INFO: `/v3/users/fd4fb85c3fcd4317ad287d5f01cb87b1` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:18.460890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4fb85c3fcd4317ad287d5f01cb87b1)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:18.463309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:18.669909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4fb85c3fcd4317ad287d5f01cb87b1'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.677045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4fb85c3fcd4317ad287d5f01cb87b1'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.677577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4fb85c3fcd4317ad287d5f01cb87b1'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.678197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4fb85c3fcd4317ad287d5f01cb87b1'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.838478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4fb85c3fcd4317ad287d5f01cb87b1'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:18.862859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1009/2010] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:18 2022] DELETE /identity/v3/users/fd4fb85c3fcd4317ad287d5f01cb87b1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:19.024109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:19.341955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:19.373969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:19.375427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:19.389697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:19.391127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:19.413741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc7660-22d0-48c0-aaa3-469c87056aec admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:19.475073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cd757-7485-4e10-8e3b-4f091f16594d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:19.501542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:19.502518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:19.503436 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:19.503932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] PATH_INFO: `/v3/projects/4e82e56d71674a038b8dd22191bcb5d7` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:19.505599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e82e56d71674a038b8dd22191bcb5d7)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:19.507956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:19.563603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:19.564169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:19.564620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:19.564930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] PATH_INFO: `/v3/users/b71d345fb1d14786b5bff255c017cf2f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:19.567863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71d345fb1d14786b5bff255c017cf2f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:19.570277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:19.644802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e82e56d71674a038b8dd22191bcb5d7', 'request_id': 'req-195e7da0-19b7-4107-a5a9-97b357877082'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.653790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e82e56d71674a038b8dd22191bcb5d7', 'request_id': 'req-195e7da0-19b7-4107-a5a9-97b357877082'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.663094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e82e56d71674a038b8dd22191bcb5d7', 'request_id': 'req-195e7da0-19b7-4107-a5a9-97b357877082'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.825250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] The token cache is being invalidate because project 4e82e56d71674a038b8dd22191bcb5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:19.826842 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.827150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-195e7da0-19b7-4107-a5a9-97b357877082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.828754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1010/2011] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:19 2022] DELETE /identity/v3/projects/4e82e56d71674a038b8dd22191bcb5d7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:19.833914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9027918-9169-41f4-abac-baa49dea59e6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:19.834810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9027918-9169-41f4-abac-baa49dea59e6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:19.835152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9027918-9169-41f4-abac-baa49dea59e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:19.839258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71d345fb1d14786b5bff255c017cf2f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.840296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71d345fb1d14786b5bff255c017cf2f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.847886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71d345fb1d14786b5bff255c017cf2f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.858761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d9027918-9169-41f4-abac-baa49dea59e6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:19.860062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d9027918-9169-41f4-abac-baa49dea59e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:19.923253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1011/2012] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:19.931169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:19.931482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:19.931826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:19.944774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71d345fb1d14786b5bff255c017cf2f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.952685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:19.954271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:19.958416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-51fb2a53-ae90-4890-baa1-58b71acc7ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71d345fb1d14786b5bff255c017cf2f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:19.965656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1012/2013] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:19.969478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1002/2014] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:19 2022] DELETE /identity/v3/users/b71d345fb1d14786b5bff255c017cf2f => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:19.970077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:19.970643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:19.970933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:19.971392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:19.980371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cae408-7e1b-4e9a-b329-f28e8393dcc5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:19.993722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:19.995206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:20.038289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:20.055613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.056348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.056631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.056934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.057949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:20.057949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.061321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.077516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1013/2015] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:20.078385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.083028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1003/2016] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:20.093710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.095740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:20.096639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef4127-a8fa-49f8-8bba-ed08f3001ee7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.102112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.103018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.103488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.103796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.104483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.106357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.193927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d306e8c-286a-4827-bebf-5093e248edaf None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.198681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.199694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.200045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.200495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.215091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.219279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.230287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1004/2017] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:20.266721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1014/2018] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:20 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:20.279521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.294853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba074e6e-ed7e-4ce4-a3f6-b05af619e977 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.370944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.371448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.371728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.372002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.372614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.375007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.514483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1005/2019] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:20 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:20.518575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.533124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4fead8-5fea-4db0-8f77-94b160e16c79 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.538712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.539339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.539657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.539895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.540546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.542245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.554377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1015/2020] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:20.557980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=7,keystone:SELECT=85,keystone:INSERT=8 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:20.559667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.574124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec15304-5904-46a8-9adb-619234631721 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.580279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.581680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.582264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.582721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.585275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.588794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.609491 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1006/2021] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:20.610708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.625430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828e301-f9d9-43c8-8103-0c723dffdcd0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.629851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.630797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.631023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.631291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] PATH_INFO: `/v3/projects/7747bd33289d4fad917c035805e9000a/users/510567ac31504635b01a27159234e6bf/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.654591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7747bd33289d4fad917c035805e9000a, user_id=510567ac31504635b01a27159234e6bf, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.657565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a593cd8-c8b0-4a5a-ab06-2be43d857c84 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.732360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:20.735106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:20.762498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f0d56e-3617-438d-a189-ee47c63fce53 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:20.778992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1016/2022] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:20 2022] PUT /identity/v3/projects/7747bd33289d4fad917c035805e9000a/users/510567ac31504635b01a27159234e6bf/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:20.795850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.799314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.799589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.907046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:20.907942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:20.908398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:20.908908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] PATH_INFO: `/v3/projects/428e204326024f038328718ccd256144` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:20.911208 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428e204326024f038328718ccd256144)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:20.916150 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:20.921816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] MFA Rules not processed for user `510567ac31504635b01a27159234e6bf`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:20.923401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:21.064390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428e204326024f038328718ccd256144', 'request_id': 'req-a8cee381-14bb-4745-97bf-1c504d895eb0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.109241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1017/2023] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:21.114577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.128295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5dfb0-f471-4eba-b961-0d54267b84ba None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:21.163891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428e204326024f038328718ccd256144', 'request_id': 'req-a8cee381-14bb-4745-97bf-1c504d895eb0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.178094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428e204326024f038328718ccd256144', 'request_id': 'req-a8cee381-14bb-4745-97bf-1c504d895eb0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.198513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] The token cache is being invalidate because project 428e204326024f038328718ccd256144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:21.198724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.199464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.204124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1007/2024] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:20 2022] DELETE /identity/v3/projects/428e204326024f038328718ccd256144 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:21.205610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.207646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:21.217046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:21.238660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:21.239381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:21.239710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:21.239992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:21.241138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:21.243631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:21.324679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cee381-14bb-4745-97bf-1c504d895eb0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:21.331779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:21.332548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:21.333095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:21.333791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] PATH_INFO: `/v3/users/74a30bc7aa88415c86fb374258e23cd1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:21.335453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a30bc7aa88415c86fb374258e23cd1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:21.338073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:21.381299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1018/2025] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:21 2022] POST /identity/v3/projects => generated 373 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:21.385377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.389092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:21.498357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b8811-91e8-4937-9321-0956b3ad42f0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:21.514551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:21.515414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:21.515598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:21.516274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:21.516980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:21.518932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:21.598136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a30bc7aa88415c86fb374258e23cd1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.599125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a30bc7aa88415c86fb374258e23cd1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.616550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a30bc7aa88415c86fb374258e23cd1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.683938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1019/2026] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:21 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:21.687891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.691601 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:21.704964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac9277-5ae4-469b-a633-ad4e606cb701 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:21.710124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:21.710953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:21.711644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:21.711925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:21.712328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a30bc7aa88415c86fb374258e23cd1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.713288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:21.715579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:21.738012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1020/2027] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:21.743144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.746653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a30bc7aa88415c86fb374258e23cd1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:21.754840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1008/2028] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:21 2022] DELETE /identity/v3/users/74a30bc7aa88415c86fb374258e23cd1 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:21.758012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4038e539-3444-4194-967b-412d8a1c0d4a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:21.835283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:21.836078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:21.836360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:21.836629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] PATH_INFO: `/v3/projects/ad739da32bc1410e9bba8124af520f3a/users/83a3409223c647cba11cb191ca8a770c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:21.859900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad739da32bc1410e9bba8124af520f3a, user_id=83a3409223c647cba11cb191ca8a770c, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:21.863788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:21.983865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1021/2029] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:21 2022] PUT /identity/v3/projects/ad739da32bc1410e9bba8124af520f3a/users/83a3409223c647cba11cb191ca8a770c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:21.987694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:21.998744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423070b3-c4d1-4295-8757-bafca849db7c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.067222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.067962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.068178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.068454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.069375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.071032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a932ca-058b-4747-b4ae-e6d45f6d2206 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:22.083059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1009/2030] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:22.087399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:22.098827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a902c-9380-4857-ba05-ff68c464f528 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.103979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.105313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.106211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.115439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] PATH_INFO: `/v3/projects/ad739da32bc1410e9bba8124af520f3a/users/83a3409223c647cba11cb191ca8a770c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.115843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad739da32bc1410e9bba8124af520f3a, user_id=83a3409223c647cba11cb191ca8a770c, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.119117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:22.131732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1022/2031] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:22 2022] PUT /identity/v3/projects/ad739da32bc1410e9bba8124af520f3a/users/83a3409223c647cba11cb191ca8a770c/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:22.144489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.144489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.144857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.185425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] MFA Rules not processed for user `83a3409223c647cba11cb191ca8a770c`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:22.187843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.264720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1010/2032] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:22.275760 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:22.277586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.345527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1822b31-5ca5-4689-838f-e26d11c5439c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.349374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.349969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.350256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.350539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.352327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.416539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.418242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33feb6bb-0092-4ffb-9a23-90187feca2f0 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:22.418374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:22.419983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.433025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1023/2033] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:22.493627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b0bd1-9e87-485e-8e31-3fec4e52ccd0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.498910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.499655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.499957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.500445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] PATH_INFO: `/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.502434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:22.503386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0c5b7a3e644101ab86cabd806cbd71)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.505453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:22.667983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0c5b7a3e644101ab86cabd806cbd71', 'request_id': 'req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:22.781067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0c5b7a3e644101ab86cabd806cbd71', 'request_id': 'req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:22.788246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0c5b7a3e644101ab86cabd806cbd71', 'request_id': 'req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:22.799867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] The token cache is being invalidate because project 6d0c5b7a3e644101ab86cabd806cbd71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:22.800162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:22.800855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b744a2b8-f4c3-46c4-82eb-336c1dd1bfac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:22.802473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1011/2034] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:22 2022] DELETE /identity/v3/projects/6d0c5b7a3e644101ab86cabd806cbd71 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:22.809279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.809769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.809966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.831029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:22.832257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.896815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:22.904358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1024/2035] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:22.907148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.907433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.911974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.928964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:22.930252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:22.938898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1012/2036] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:22.943291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:22.953308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607828-0417-4b05-8933-c9add4cf4c60 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.958801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.959419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.959691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.959965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.960619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:22.960959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.963160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:22.975249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1025/2037] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:22.980551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:22.989937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb53e1e-4020-414e-bc26-74950aff98f3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:22.994559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:22.995165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:22.995443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:22.995716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:22.996345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:22.998273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.131086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1013/2038] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:22 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:23.135700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.146736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089079f-41ed-4fe9-b987-56e3769854a5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.205481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.206097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.206424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.206821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.207478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.213999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.313322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1026/2039] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:23 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:23.317266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.327925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116f122-694d-4ed1-8f21-5a1141b3e0e1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.329902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:23.333184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.333762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.334042 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.334314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.334994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.338712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.347489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1014/2040] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:23.351756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.362752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88befe8a-dfd8-4929-a67c-da14ea76fae4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.369544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.370154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.370433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.370707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.371355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.372981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314cd72-c8ab-4570-ad0f-28b15183092b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.384046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1027/2041] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:23.388940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.398978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18ad2-d7b8-46c9-8a0f-2054675651be admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.404084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.404784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.405070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.405348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] PATH_INFO: `/v3/projects/e8b263dbd441477893c5fc88cbb9ae8d/users/cc1c29e15f1042a6be0aba082ac0365a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.426980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b263dbd441477893c5fc88cbb9ae8d, user_id=cc1c29e15f1042a6be0aba082ac0365a, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.429760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.523957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1015/2042] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:23 2022] PUT /identity/v3/projects/e8b263dbd441477893c5fc88cbb9ae8d/users/cc1c29e15f1042a6be0aba082ac0365a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:23.535531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.535814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.536089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.563968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] MFA Rules not processed for user `cc1c29e15f1042a6be0aba082ac0365a`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:23.565230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:23.629594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1028/2043] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:23.635002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.701831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:23.711804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b27614-5bac-4c9f-b4ec-24f4322508ec admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.716458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.717084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.717364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.717633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.718260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.720163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:23.851681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1016/2044] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:23 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:23.856259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:23.867249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69a1d1-c90d-4d6f-b5a5-a0f5f392fb72 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:23.918135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:23.927035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:23.927638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:23.927915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:23.928193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:23.928846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:23.930735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.009107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1029/2045] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:23 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:24.012910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:24.023267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc0b92-dd03-4765-a333-a812de07bba4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:24.030053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:24.030863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.031146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.031887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.032541 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:24.034173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.050002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:24.050396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1017/2046] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:24.060666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddaf76-5d5b-4bd1-b106-4ae6794d6f30 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:24.072785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:24.073476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.073754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.074027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] PATH_INFO: `/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b/users/b74923cb3b6a40a68c49a3b78968af64/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.092521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4594e7b8eb7744a78026f71c4b8eef7b, user_id=b74923cb3b6a40a68c49a3b78968af64, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:24.095341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.181096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1030/2047] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:24 2022] PUT /identity/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b/users/b74923cb3b6a40a68c49a3b78968af64/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:24.185142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:24.195744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4323f98-aab9-4048-ad10-f73d694588f2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:24.272032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:24.272742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.273047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.273319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.274114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:24.276258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bc93e-97ae-4411-91de-5d408956ec45 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.292710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1018/2048] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:24.299863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:24.306662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4ee7c-1c18-4e74-bdb1-4543cbf271cc admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:24.311888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:24.312917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.313721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.314083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] PATH_INFO: `/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b/users/b74923cb3b6a40a68c49a3b78968af64/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.316269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4594e7b8eb7744a78026f71c4b8eef7b, user_id=b74923cb3b6a40a68c49a3b78968af64, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:24.318947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.332146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1031/2049] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:24 2022] PUT /identity/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b/users/b74923cb3b6a40a68c49a3b78968af64/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:24.343188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05755-5051-495b-86c2-b566195138f3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.343573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05755-5051-495b-86c2-b566195138f3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.343850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf05755-5051-495b-86c2-b566195138f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.375131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-daf05755-5051-495b-86c2-b566195138f3 None None] MFA Rules not processed for user `b74923cb3b6a40a68c49a3b78968af64`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:24.376507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-daf05755-5051-495b-86c2-b566195138f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:24.388572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:24.456239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1019/2050] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:24.468006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:24.469898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:24.533439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d47360-8902-422e-bfa2-e6a934129cbc admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:24.537433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:24.538095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:24.538392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:24.538664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:24.547682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:24.549136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:24.558588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1032/2051] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:25.065550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:25.476875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf05755-5051-495b-86c2-b566195138f3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:25.543709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf05755-5051-495b-86c2-b566195138f3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:25.548296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:25.548932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:25.549209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:25.549480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] PATH_INFO: `/v3/users/cc1c29e15f1042a6be0aba082ac0365a` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:25.550909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1c29e15f1042a6be0aba082ac0365a)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:25.552895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:25.794552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1c29e15f1042a6be0aba082ac0365a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:25.795206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1c29e15f1042a6be0aba082ac0365a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:25.802016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1c29e15f1042a6be0aba082ac0365a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:25.860046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:25.861479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:25.867190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1c29e15f1042a6be0aba082ac0365a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:25.875143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab43d4f-b4fe-47ca-b91e-50741434c903 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:25.880888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:25.881637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:25.881848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-816b0755-25b8-4df9-9ead-a67f6ed88ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1c29e15f1042a6be0aba082ac0365a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:25.881980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:25.885494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] PATH_INFO: `/v3/users/77eb077134f24e32affb3208f16ce583` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:25.886274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:25.890464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77eb077134f24e32affb3208f16ce583)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:25.892473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:25.893131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1020/2052] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:25 2022] DELETE /identity/v3/users/cc1c29e15f1042a6be0aba082ac0365a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:25.901339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:25.901674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:25.902877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:25.926892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:25.927239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:25.993359 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1021/2053] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:26.003199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:26.004594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.061284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e559df7-4cdd-4f06-9220-814c8f403f19 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:26.064910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:26.065514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.065804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.066070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.075334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:26.076908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69491a-e537-4134-8fcf-595ba8b7a694 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:26.086193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1022/2054] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:26.131857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77eb077134f24e32affb3208f16ce583'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.139805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77eb077134f24e32affb3208f16ce583'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.140801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77eb077134f24e32affb3208f16ce583'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.141140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77eb077134f24e32affb3208f16ce583'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.209470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77eb077134f24e32affb3208f16ce583'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.230406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1033/2055] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:25 2022] DELETE /identity/v3/users/77eb077134f24e32affb3208f16ce583 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:26.236708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.236997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.237274 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.268198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:26.270858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.288321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:26.404763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1023/2056] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:26.413392 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:26.415101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.494446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c36874-8e68-4a88-ad0f-2b5875265594 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:26.500852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:26.501505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.501783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.502051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.513430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:26.514945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:26.525632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1034/2057] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:26.599882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:26.601384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.613002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf84e61-6d3d-4640-86d0-0ec248332b82 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:26.617579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:26.618157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.618971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.619240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] PATH_INFO: `/v3/users/9a7cf8757cfa434e81b3dac1c05f66ba` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.620354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7cf8757cfa434e81b3dac1c05f66ba)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:26.622295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:26.646309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:26.646502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.666617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb6b7b-b0a2-446b-90a9-4c36e993ed95 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:26.667065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:26.675196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:26.675196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.675196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.675196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] PATH_INFO: `/v3/projects/e8b263dbd441477893c5fc88cbb9ae8d` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.679148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b263dbd441477893c5fc88cbb9ae8d)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:26.683627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:26.853323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7cf8757cfa434e81b3dac1c05f66ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.854045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7cf8757cfa434e81b3dac1c05f66ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.858048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b263dbd441477893c5fc88cbb9ae8d', 'request_id': 'req-237f3337-3225-4d77-9f31-182a0366049a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.862349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7cf8757cfa434e81b3dac1c05f66ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.864759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b263dbd441477893c5fc88cbb9ae8d', 'request_id': 'req-237f3337-3225-4d77-9f31-182a0366049a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.874952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b263dbd441477893c5fc88cbb9ae8d', 'request_id': 'req-237f3337-3225-4d77-9f31-182a0366049a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.918173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7cf8757cfa434e81b3dac1c05f66ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.930898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb483700-07d4-4732-aa23-684380b84e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7cf8757cfa434e81b3dac1c05f66ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.937466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1024/2058] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:26 2022] DELETE /identity/v3/users/9a7cf8757cfa434e81b3dac1c05f66ba => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:26.941954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:26.942241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:26.942509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:26.965946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:26.967434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:26.985962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] The token cache is being invalidate because project e8b263dbd441477893c5fc88cbb9ae8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:26.986354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.987063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:26.991984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1035/2059] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:26 2022] DELETE /identity/v3/projects/e8b263dbd441477893c5fc88cbb9ae8d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:26.992956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:26.996836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.043200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1025/2060] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:27.043200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:27.043671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.058038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af174fe8-e718-4d53-82fd-6e77b5d3912d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.066000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.066969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.067449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.067930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] PATH_INFO: `/v3/users/510567ac31504635b01a27159234e6bf` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.070021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] RBAC: Authorizing `identity:delete_user(user_id=510567ac31504635b01a27159234e6bf)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.070869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:27.074591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:27.090875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f3337-3225-4d77-9f31-182a0366049a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.096680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.099002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.099002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.099002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] PATH_INFO: `/v3/users/b74923cb3b6a40a68c49a3b78968af64` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.099569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74923cb3b6a40a68c49a3b78968af64)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.101563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:27.269753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '510567ac31504635b01a27159234e6bf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.270520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '510567ac31504635b01a27159234e6bf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.277398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '510567ac31504635b01a27159234e6bf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.366084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74923cb3b6a40a68c49a3b78968af64'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.374191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74923cb3b6a40a68c49a3b78968af64'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.374980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74923cb3b6a40a68c49a3b78968af64'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.375669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74923cb3b6a40a68c49a3b78968af64'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.380680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '510567ac31504635b01a27159234e6bf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.394020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '510567ac31504635b01a27159234e6bf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.399980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1026/2061] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:27 2022] DELETE /identity/v3/users/510567ac31504635b01a27159234e6bf => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:27.403865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:27.405270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.462153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab04023-c7d8-4f7e-a06f-a7b9b29c207e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.465930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.466534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.466815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.467087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.468880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.470272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74923cb3b6a40a68c49a3b78968af64'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.489302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1036/2062] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:26 2022] DELETE /identity/v3/users/b74923cb3b6a40a68c49a3b78968af64 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:27.493224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:27.495569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.536918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.538375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb370d6-7935-4abe-afe8-0c990c9bf277 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:27.552133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1027/2063] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:27.562184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.562184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.562184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.575992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cdd51-f0bf-4ac3-8570-6f7f69fa4783 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.576209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:27.577350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.581128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.581812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.582215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.582362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] PATH_INFO: `/v3/projects/8359395fb8fc4967a446471664d6f4f0` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.582757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:27.583645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8359395fb8fc4967a446471664d6f4f0)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.586334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:27.597248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1028/2064] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:27.600946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:27.603610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:27.674501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d11db-806a-488b-b9fd-e2f36b26b19e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.678543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.679157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.679439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.679711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.690739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.692385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:27.703391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1029/2065] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:27.777774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8359395fb8fc4967a446471664d6f4f0', 'request_id': 'req-4946ddff-a774-48ee-bee1-b959b1a3d922'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.790430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8359395fb8fc4967a446471664d6f4f0', 'request_id': 'req-4946ddff-a774-48ee-bee1-b959b1a3d922'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.798335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8359395fb8fc4967a446471664d6f4f0', 'request_id': 'req-4946ddff-a774-48ee-bee1-b959b1a3d922'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.904920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] The token cache is being invalidate because project 8359395fb8fc4967a446471664d6f4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:27.905459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.906147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:27.908634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1037/2066] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:27 2022] DELETE /identity/v3/projects/8359395fb8fc4967a446471664d6f4f0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:27.913538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:27.928180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438f1f0-a83a-4fb2-848c-2adf8313d8e7 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:27.949627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:27.986888 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:27.987523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:27.987805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:27.988078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] PATH_INFO: `/v3/users/9a94e4fef3d94bb5bf6ecda16dba30fd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:27.989259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a94e4fef3d94bb5bf6ecda16dba30fd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:27.991270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:28.004582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:28.211134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:28.217810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.226733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946ddff-a774-48ee-bee1-b959b1a3d922 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:28.235777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:28.236621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:28.237526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:28.237959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] PATH_INFO: `/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:28.240766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4594e7b8eb7744a78026f71c4b8eef7b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:28.243080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:28.254519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a94e4fef3d94bb5bf6ecda16dba30fd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.254519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a94e4fef3d94bb5bf6ecda16dba30fd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.268948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a94e4fef3d94bb5bf6ecda16dba30fd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.356090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a94e4fef3d94bb5bf6ecda16dba30fd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.368597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a94e4fef3d94bb5bf6ecda16dba30fd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.374447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1030/2067] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:27 2022] DELETE /identity/v3/users/9a94e4fef3d94bb5bf6ecda16dba30fd => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:28.375305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:28.376568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.419196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4594e7b8eb7744a78026f71c4b8eef7b', 'request_id': 'req-e3624029-4b58-4703-b29b-c1e23403321d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.425404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4594e7b8eb7744a78026f71c4b8eef7b', 'request_id': 'req-e3624029-4b58-4703-b29b-c1e23403321d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.432827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4594e7b8eb7744a78026f71c4b8eef7b', 'request_id': 'req-e3624029-4b58-4703-b29b-c1e23403321d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.465074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5d488-2ba4-425b-9093-14bf1a3f03eb admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:28.521804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:28.522428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:28.522703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:28.522971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] PATH_INFO: `/v3/projects/61f114eb72cf4c9db994ace924c98c96` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:28.524168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f114eb72cf4c9db994ace924c98c96)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:28.526874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:28.538693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] The token cache is being invalidate because project 4594e7b8eb7744a78026f71c4b8eef7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:28.538985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.539624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.541343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1038/2068] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:28 2022] DELETE /identity/v3/projects/4594e7b8eb7744a78026f71c4b8eef7b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:28.544615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:28.544615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.556748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624029-4b58-4703-b29b-c1e23403321d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:28.561757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:28.562390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:28.562667 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:28.562941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] PATH_INFO: `/v3/projects/7747bd33289d4fad917c035805e9000a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:28.565122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7747bd33289d4fad917c035805e9000a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:28.566048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:28.744399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f114eb72cf4c9db994ace924c98c96', 'request_id': 'req-ea2958c6-c225-4757-b954-333ce7ef105d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.745103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7747bd33289d4fad917c035805e9000a', 'request_id': 'req-939af605-7175-4bf4-b152-00c39525989e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.751187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7747bd33289d4fad917c035805e9000a', 'request_id': 'req-939af605-7175-4bf4-b152-00c39525989e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.758148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7747bd33289d4fad917c035805e9000a', 'request_id': 'req-939af605-7175-4bf4-b152-00c39525989e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.826308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f114eb72cf4c9db994ace924c98c96', 'request_id': 'req-ea2958c6-c225-4757-b954-333ce7ef105d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.834340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f114eb72cf4c9db994ace924c98c96', 'request_id': 'req-ea2958c6-c225-4757-b954-333ce7ef105d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.838810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:28.847309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] The token cache is being invalidate because project 61f114eb72cf4c9db994ace924c98c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:28.847725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.848395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ea2958c6-c225-4757-b954-333ce7ef105d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.850211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1031/2069] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:28 2022] DELETE /identity/v3/projects/61f114eb72cf4c9db994ace924c98c96 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:28.851718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:28.851878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:28.852151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:28.875276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:28.876808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.890529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] The token cache is being invalidate because project 7747bd33289d4fad917c035805e9000a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:28.891220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.891742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:28.894483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1039/2070] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:28 2022] DELETE /identity/v3/projects/7747bd33289d4fad917c035805e9000a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:28.895452 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:28.897884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.940786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1032/2071] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:28.941732 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:28.943251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:28.955148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff72568-c11f-40fc-9f1f-397658233cf0 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:28.959990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:28.960621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:28.960931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:28.961206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] PATH_INFO: `/v3/users/428e8c2d73b9444c82a7f721828dddd4` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:28.962999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=428e8c2d73b9444c82a7f721828dddd4)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:28.965075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:28.974818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:29.001733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939af605-7175-4bf4-b152-00c39525989e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.006388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.007027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.007306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.007579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] PATH_INFO: `/v3/projects/db0352401e7e414fba9ccef65acf8e4b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.008795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0352401e7e414fba9ccef65acf8e4b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.011209 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:29.154387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0352401e7e414fba9ccef65acf8e4b', 'request_id': 'req-4b76a621-24c4-4b1c-888c-f44ebdcacc73'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.154858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '428e8c2d73b9444c82a7f721828dddd4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.155510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '428e8c2d73b9444c82a7f721828dddd4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.160197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0352401e7e414fba9ccef65acf8e4b', 'request_id': 'req-4b76a621-24c4-4b1c-888c-f44ebdcacc73'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.162550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '428e8c2d73b9444c82a7f721828dddd4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.166737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0352401e7e414fba9ccef65acf8e4b', 'request_id': 'req-4b76a621-24c4-4b1c-888c-f44ebdcacc73'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.254730 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '428e8c2d73b9444c82a7f721828dddd4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.267168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '428e8c2d73b9444c82a7f721828dddd4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.273158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1033/2072] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:28 2022] DELETE /identity/v3/users/428e8c2d73b9444c82a7f721828dddd4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:29.273945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:29.275308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.314287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] The token cache is being invalidate because project db0352401e7e414fba9ccef65acf8e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:29.316551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.316551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4b76a621-24c4-4b1c-888c-f44ebdcacc73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.317969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1040/2073] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:28 2022] DELETE /identity/v3/projects/db0352401e7e414fba9ccef65acf8e4b => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:29.319730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.320249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.321124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.342155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718f84-79d7-43ea-9c27-98c97bd10dd4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.345370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:29.346993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.350452 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.351164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.351578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.351690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] PATH_INFO: `/v3/users/83a3409223c647cba11cb191ca8a770c` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.353686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a3409223c647cba11cb191ca8a770c)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.355615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:29.357891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1041/2074] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:29.362480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:29.362480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.375062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f2751-43ad-4a5a-8638-2e5e96306baa None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.388567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.389295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.389575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.389850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] PATH_INFO: `/v3/users/ac2848a6d33d40d4bc7f16514ddce3d3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.391130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2848a6d33d40d4bc7f16514ddce3d3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.393012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:29.507369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:29.545976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a3409223c647cba11cb191ca8a770c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.546658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a3409223c647cba11cb191ca8a770c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.554468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a3409223c647cba11cb191ca8a770c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.565455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:29.628156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2848a6d33d40d4bc7f16514ddce3d3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.635141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2848a6d33d40d4bc7f16514ddce3d3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.635815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2848a6d33d40d4bc7f16514ddce3d3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.636430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2848a6d33d40d4bc7f16514ddce3d3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.637589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a3409223c647cba11cb191ca8a770c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.650800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a3409223c647cba11cb191ca8a770c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.656788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1034/2075] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:29 2022] DELETE /identity/v3/users/83a3409223c647cba11cb191ca8a770c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:29.657655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:29.659059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.725923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c32f33-5b23-4513-a179-739798534436 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.731448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.732072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.732351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.732623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.733379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:29.733636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.735292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:29.750302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2848a6d33d40d4bc7f16514ddce3d3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:29.752301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1035/2076] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:29.752798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:29.754492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.774377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041634e-942e-4c9f-901f-6dd64b662e6e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.775702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.776350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.776727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.777012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] PATH_INFO: `/v3/users/86030a2f71844c9e9ab894ef3955e5c0` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.778845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1042/2077] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:29 2022] DELETE /identity/v3/users/ac2848a6d33d40d4bc7f16514ddce3d3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:29.779243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] RBAC: Authorizing `identity:delete_user(user_id=86030a2f71844c9e9ab894ef3955e5c0)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.779834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:29.781470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:29.781703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:29.852330 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411cb3f-38d9-4c4f-b153-908761e7f38e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:29.857152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:29.857790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:29.858072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:29.858346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] PATH_INFO: `/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:29.860545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2c6ce86ba84b41a3afb2566fa280b8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:29.861405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.031227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2c6ce86ba84b41a3afb2566fa280b8', 'request_id': 'req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.040781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2c6ce86ba84b41a3afb2566fa280b8', 'request_id': 'req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.044172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86030a2f71844c9e9ab894ef3955e5c0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.045096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86030a2f71844c9e9ab894ef3955e5c0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.061228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86030a2f71844c9e9ab894ef3955e5c0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.064273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2c6ce86ba84b41a3afb2566fa280b8', 'request_id': 'req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.176570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86030a2f71844c9e9ab894ef3955e5c0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.192413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86030a2f71844c9e9ab894ef3955e5c0'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.201821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1036/2078] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:29 2022] DELETE /identity/v3/users/86030a2f71844c9e9ab894ef3955e5c0 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:30.201821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.203128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.271521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] The token cache is being invalidate because project aa2c6ce86ba84b41a3afb2566fa280b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:30.272060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.272192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc29bc-c9e5-493e-b920-75ad396f197e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.279721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.279721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1043/2079] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:29 2022] DELETE /identity/v3/projects/aa2c6ce86ba84b41a3afb2566fa280b8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:30.279721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.279721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.280039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.280039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.280039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.280039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.280309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.282181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764f4b0-3193-49f0-b402-a79853fe76db admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.347146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8c276-7785-4cb2-9a1d-88ae23c51e2e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.352680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.353337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.353612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.353882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] PATH_INFO: `/v3/projects/ad739da32bc1410e9bba8124af520f3a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.355012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad739da32bc1410e9bba8124af520f3a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.357021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.390017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1037/2080] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:30.390017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5030-9a1a-45ce-9897-95b9ee3e119b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.390017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5030-9a1a-45ce-9897-95b9ee3e119b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.390017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5030-9a1a-45ce-9897-95b9ee3e119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.408935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-911a5030-9a1a-45ce-9897-95b9ee3e119b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:30.410136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-911a5030-9a1a-45ce-9897-95b9ee3e119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.475085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1038/2081] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:30.475085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.475085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.475085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.493259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad739da32bc1410e9bba8124af520f3a', 'request_id': 'req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.494549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:30.495748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.499449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad739da32bc1410e9bba8124af520f3a', 'request_id': 'req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.505623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad739da32bc1410e9bba8124af520f3a', 'request_id': 'req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.523373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:30.554095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1039/2082] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:30.556419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.556758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.568034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a8fce9-39a7-474d-b0f0-7413cb2bc374 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.573130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.573676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.573949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.574216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] PATH_INFO: `/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.579049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1bbc0ffdae4b6abb217be9b517e5a1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.579049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.580688 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] The token cache is being invalidate because project ad739da32bc1410e9bba8124af520f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:30.581015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.581660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.583282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1044/2083] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:30 2022] DELETE /identity/v3/projects/ad739da32bc1410e9bba8124af520f3a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:30.584127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.585581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.599136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be37c3-ee3c-424d-b436-1ecbd9e29dc9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.604431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.607680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.607680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.607680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.607680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.608105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.717816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1045/2084] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:30.718651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.720207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.730453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1bbc0ffdae4b6abb217be9b517e5a1', 'request_id': 'req-343b2c59-a07e-46b6-acf7-986514d5b369'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.738742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:30.784770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dea20-4b0b-4681-8783-44f96bf3e488 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.820841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1bbc0ffdae4b6abb217be9b517e5a1', 'request_id': 'req-343b2c59-a07e-46b6-acf7-986514d5b369'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.827625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1bbc0ffdae4b6abb217be9b517e5a1', 'request_id': 'req-343b2c59-a07e-46b6-acf7-986514d5b369'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.833658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.834299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.834634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.834847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.836625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.839420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] The token cache is being invalidate because project 5b1bbc0ffdae4b6abb217be9b517e5a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:30.842896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.842896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-343b2c59-a07e-46b6-acf7-986514d5b369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:30.842896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1040/2085] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:30 2022] DELETE /identity/v3/projects/5b1bbc0ffdae4b6abb217be9b517e5a1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:30.843459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.843736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.844005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.864269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:30.865525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.906770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.908218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2820b1f-82aa-4b4a-acfc-b48d5eda859e service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.918905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1046/2086] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:30.920147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981bce-991a-4d3c-8e9e-3f245f26a48f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.920432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981bce-991a-4d3c-8e9e-3f245f26a48f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.920805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e981bce-991a-4d3c-8e9e-3f245f26a48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.929356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1041/2087] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:30.929356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.930960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.941226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec80f3-727a-4a1b-9941-c0a3bfb1c164 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.943506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0e981bce-991a-4d3c-8e9e-3f245f26a48f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:30.944791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e981bce-991a-4d3c-8e9e-3f245f26a48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.946357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.947241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.947241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.947496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.948146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.949800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.955734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1047/2088] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:30.956022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.956312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.956894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.968816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1042/2089] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:30.968816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:30.979905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576034b2-4925-46d3-840a-a247f29804bf admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:30.985623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:30.985821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:30.986647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:30.986803 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:30.987092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:30.987467 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:30.988487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:30.988955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:30.990717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04126b0-f3be-40a5-870a-fc4467500d04 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:30.997020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1048/2090] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:30.998394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.008666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f711a-3713-4047-a6c3-5c682a08d33f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.012395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1043/2091] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:31.012395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.013389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.013389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.014252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.014861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.015137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.015408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.016217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.017794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623ba7f-8d9f-45de-9dfc-2455f3290a27 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.031902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1049/2092] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:31.031902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.031902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.031902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.034584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:31.035825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.044972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1044/2093] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:31.045765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.054959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:31.056157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96258d07-4da1-4d90-bbcd-2871a498209a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.056273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.060838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.061463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.061740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.062015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.062643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.067574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.067710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1050/2094] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:31.067710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.076320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadb26cb-3a9f-4122-aa78-013888b28244 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.080898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.081542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.081819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.082089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] PATH_INFO: `/v3/projects/0b4e12b2521a40ae8aecd5c818bfc623/users/379fe93540ee4318be1b10dbcc417148/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.097225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4e12b2521a40ae8aecd5c818bfc623, user_id=379fe93540ee4318be1b10dbcc417148, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.099874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.161805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1045/2095] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:31.162951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.209192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1051/2096] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:31 2022] PUT /identity/v3/projects/0b4e12b2521a40ae8aecd5c818bfc623/users/379fe93540ee4318be1b10dbcc417148/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:31.214005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.231698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654557-32e8-4159-9b9b-4424f6abf9dc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.292839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc222-1b5f-43ee-8641-d9531ec3beb1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.297657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.298269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.298548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.298815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.299493 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:31.299797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.301432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.307761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.309310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.309767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.310644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.311576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:31.311882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.313550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9971d1-f4ec-4d8c-be5e-c292d1d40c6b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.314614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1052/2097] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:31.315655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.326793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa0a34-c077-4b9b-9123-ba0d8980b567 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.327166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1046/2098] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:31.328533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.328855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.329132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.337581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.338432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.338815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.339171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.339863 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.341921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.350192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:31.351505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.364831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1047/2099] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:31.364831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.383416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f1dad-e3a8-4164-9a56-02044785d981 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.390382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.391028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.391304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.399929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.399929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.399929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.424746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1053/2100] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:31.426745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.434569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:31.438463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ef34-c826-478c-aa56-b7489938e45b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.443516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.444063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.444341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.444612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.445271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.447755 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.483084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1048/2101] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:31.483803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.485419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.549065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e26e-41fa-4fcd-b798-cb1c1656edee admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.554293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.554930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.555211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.555485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] PATH_INFO: `/v3/projects/ffda9549a4db451a831cbb21923d5e38` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.556674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffda9549a4db451a831cbb21923d5e38)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.558594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.588890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1054/2102] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:31.589706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.591302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.593579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:31.647028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530f504-ba73-494d-b111-28482f8f2e08 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.650662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.651259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.651533 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.651805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.689057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffda9549a4db451a831cbb21923d5e38', 'request_id': 'req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:31.717812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.719235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.728712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1055/2103] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:31.729499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.738251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cb736-c628-478e-b376-525dbdb67156 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.794833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.795440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.795721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.795993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.796730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.798340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.808561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1056/2104] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:31.809352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.818628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f107987-2c6c-4b9a-9a9c-f9ae9ec64487 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.823984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.824582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.824888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.825163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.825765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.827617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.883614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffda9549a4db451a831cbb21923d5e38', 'request_id': 'req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:31.891245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffda9549a4db451a831cbb21923d5e38', 'request_id': 'req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:31.902065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] The token cache is being invalidate because project ffda9549a4db451a831cbb21923d5e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:31.903058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:31.903058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:31.904595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1049/2105] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:31 2022] DELETE /identity/v3/projects/ffda9549a4db451a831cbb21923d5e38 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:31.907661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.907661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.919579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4f3d-1eb3-4f9e-b06d-dc46b9a43f05 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.924245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.924882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.925163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.925440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.926067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.928020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.964963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1057/2106] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:31.964963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:31.964963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:31.976779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a842217-cfc1-4360-a63f-878c54744a89 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:31.982884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:31.983517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:31.983801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:31.984077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:31.984757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:31.986441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:31.997785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1058/2107] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:31.998600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.002083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.014669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5d708-add2-4955-84bf-3b5853b309c7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.019319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.021157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.021157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.021157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] PATH_INFO: `/v3/users/c5f729e00a8c431b80015b9a22349ae4` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.021672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f729e00a8c431b80015b9a22349ae4)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.023600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.048691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1050/2108] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:31 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:32.049140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.050931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.064001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e734e-cee4-4322-9c2e-ce680c1b7df5 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.069056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.069785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.070066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.070343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.070996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.072639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.085179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1051/2109] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:32.086477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.097880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022b8b7-11f6-4730-9714-6326e15848ba admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.101694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.102907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.103545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.103685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] PATH_INFO: `/v3/projects/033ea0bc60d544bf9a7ba8abbafafa0f/users/ceb09f6e4b1e48c2a84096c36cdb2c2e/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.120577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033ea0bc60d544bf9a7ba8abbafafa0f, user_id=ceb09f6e4b1e48c2a84096c36cdb2c2e, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.123603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.249278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1052/2110] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:32 2022] PUT /identity/v3/projects/033ea0bc60d544bf9a7ba8abbafafa0f/users/ceb09f6e4b1e48c2a84096c36cdb2c2e/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:32.250119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.287376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f729e00a8c431b80015b9a22349ae4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:32.296835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f729e00a8c431b80015b9a22349ae4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:32.302768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f729e00a8c431b80015b9a22349ae4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:32.302768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f729e00a8c431b80015b9a22349ae4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:32.324153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b143c-932a-4795-926f-ed73d4e84654 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.329771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.331242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.331242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.331242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.331572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.333196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.345318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1053/2111] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:32.346368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.349319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.362091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffe759-2675-42ff-9d87-26dc2fb26ca4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.366915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.367510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.367782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.368590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.369286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.370863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da01d21-d5e3-448f-8c54-3fb1384fdf78 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.381015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1054/2112] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:32.382222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.382499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.382762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.387620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f729e00a8c431b80015b9a22349ae4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:32.418219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1059/2113] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:31 2022] DELETE /identity/v3/users/c5f729e00a8c431b80015b9a22349ae4 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:32.420073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.423640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.433305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:32.439658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] MFA Rules not processed for user `ceb09f6e4b1e48c2a84096c36cdb2c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:32.441020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.450233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:32.508102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6ff3-2542-40c5-aac8-f6503f536121 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.515304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.515925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.517306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.517615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.518302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.521241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.542217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1060/2114] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:32.542217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.546877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1055/2115] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:32.547643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.553084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0010b-df17-4856-9eb0-59f2a3567488 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.558393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.559203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886d185-6152-42ef-9b08-13c4c61e4ed5 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.559357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.559736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.560240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] PATH_INFO: `/v3/projects/19de93aae58940babd61ffbf31f45c77/users/393f109857044c84920a91826ebadb9d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.566583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.571073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.571073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.571073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] PATH_INFO: `/v3/projects/647a6443ec0c49e58d3cdb373d681c38/users/ed5bde75f6d2477ab2747016ab5c309e/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.579791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19de93aae58940babd61ffbf31f45c77, user_id=393f109857044c84920a91826ebadb9d, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.583235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.589619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] RBAC: Authorizing `identity:create_grant(project_id=647a6443ec0c49e58d3cdb373d681c38, user_id=ed5bde75f6d2477ab2747016ab5c309e, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.593399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a71b0-8014-4f6e-8748-39353b68557e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.656260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1061/2116] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:32 2022] PUT /identity/v3/projects/19de93aae58940babd61ffbf31f45c77/users/393f109857044c84920a91826ebadb9d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:32.657832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.659658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.707421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1056/2117] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:32 2022] PUT /identity/v3/projects/647a6443ec0c49e58d3cdb373d681c38/users/ed5bde75f6d2477ab2747016ab5c309e/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:32.708833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73a2e-4838-45eb-8445-599eb5135794 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.717852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73a2e-4838-45eb-8445-599eb5135794 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.717852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce73a2e-4838-45eb-8445-599eb5135794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.749158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7ce73a2e-4838-45eb-8445-599eb5135794 None None] MFA Rules not processed for user `393f109857044c84920a91826ebadb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:32.753533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7ce73a2e-4838-45eb-8445-599eb5135794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.797485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83aebc-4de0-4fa3-aaf9-530bc8a24188 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.849223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1057/2118] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:32.851050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.851050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.851593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.871037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:32.872247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.874633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.875273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.875553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.875829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.876453 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.878403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.879936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1058/2119] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:32.883167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.883167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:32.939845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1763b-41b6-4b8c-a085-cc641be58108 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:32.943613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:32.944221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:32.944496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:32.944808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:32.954137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:32.955603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:32.965847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1059/2120] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:32.966750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:32.968380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.045023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1062/2121] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:33.046490 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.048442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.077573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd26a7-e47e-4956-9a1e-c7a11e7a7dba service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.143887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8149598-e76c-4cc7-a081-74e791368e3d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.147018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.147634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.147904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.148168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.151707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.151707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.151707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.151707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] PATH_INFO: `/v3/projects/880d76edb3b54339925807870ed5ecee` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.152146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880d76edb3b54339925807870ed5ecee)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.153331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.158621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.160132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.174660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1060/2122] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:33.174660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.182105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c60df-eb63-4879-900d-853271754976 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.187831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.188427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.188735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.189004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.189626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.194388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c6b7e-7bb3-4d19-8388-b39818d1b002 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.321234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880d76edb3b54339925807870ed5ecee', 'request_id': 'req-f895089e-eeac-45c5-b4e0-a975bf4439b5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:33.333940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:33.337522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1061/2123] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:33.340389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880d76edb3b54339925807870ed5ecee', 'request_id': 'req-f895089e-eeac-45c5-b4e0-a975bf4439b5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:33.340538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.340882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.341153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.365206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880d76edb3b54339925807870ed5ecee', 'request_id': 'req-f895089e-eeac-45c5-b4e0-a975bf4439b5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:33.389095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] MFA Rules not processed for user `379fe93540ee4318be1b10dbcc417148`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:33.390324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.468094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] The token cache is being invalidate because project 880d76edb3b54339925807870ed5ecee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:33.468513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:33.469541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:33.470174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1062/2124] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:33.471165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.473351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1063/2125] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:33 2022] DELETE /identity/v3/projects/880d76edb3b54339925807870ed5ecee => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:33.479787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.481773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.566466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055b0eb-22f0-455d-88c8-d8b055eed8a3 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.570128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895089e-eeac-45c5-b4e0-a975bf4439b5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.575674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.576456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.576629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.576961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.577290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.577408 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.577870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.578083 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.578266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.578985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.581121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ace2bc-a600-4cf9-ae8f-c4a4f862ea6a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.593750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3a8a-5cbc-41cf-a091-1d1f6ba579b0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.606537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1063/2126] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:33.608602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.611227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.611227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.632255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:33.634269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.653068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1064/2127] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:33.653959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.659933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.673318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57e28e-2f79-4154-a275-1b0074fb1a9d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.685336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.686225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.686531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.686804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] PATH_INFO: `/v3/projects/09dac59775044c43b70b09d9241cbe9e/users/75835548ad15448f80ed832e7aae5a38/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.706051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dac59775044c43b70b09d9241cbe9e, user_id=75835548ad15448f80ed832e7aae5a38, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.708212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.713176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1064/2128] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:33.714365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.714703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.715006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.754180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:33.766046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:33.857677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1065/2129] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:33.858710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.870346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ab272-9d05-43a1-bda2-2f5c8e05cd60 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.875068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.875735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.876407 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.877331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.878020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.880783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.885372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1065/2130] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:33 2022] PUT /identity/v3/projects/09dac59775044c43b70b09d9241cbe9e/users/75835548ad15448f80ed832e7aae5a38/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:33.886379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:33.901161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:33.985894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742a0d-1ac3-4b94-a921-95a86a94ba64 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:33.993475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:33.994353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:33.994685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:33.994956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:33.995659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:33.995967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:33.997105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1066/2131] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:33 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:33.998017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:33.998126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.016397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61b17b-da1e-424c-9b78-4179ce4bba8f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.023011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.023742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.024027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.024367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.025239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.027083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.033075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1066/2132] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:34.034021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.045942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0dfe3-3c70-45bd-8966-d577aacb1c10 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.051330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.052075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.052357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.052830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.053494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.055106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.063812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1067/2133] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:34.064109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.070149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1067/2134] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:34.071072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.084498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b17e3-62b3-41d0-aaf4-8b3605b8e47f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.098582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9be8-9c28-4abd-9ab7-dd683938fbc4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.104200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.105280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.105705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.106267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] PATH_INFO: `/v3/projects/09dac59775044c43b70b09d9241cbe9e/users/75835548ad15448f80ed832e7aae5a38/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.107322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.108067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.108419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.109020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.109587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09dac59775044c43b70b09d9241cbe9e, user_id=75835548ad15448f80ed832e7aae5a38, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.110414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.112469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac48c3-d01c-47b2-b43e-866fd3cd59f9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.113287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.131870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1068/2135] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:34 2022] PUT /identity/v3/projects/09dac59775044c43b70b09d9241cbe9e/users/75835548ad15448f80ed832e7aae5a38/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:34.133136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.217139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95fc9-17d7-4324-8033-eeaa9a88b4a7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.222651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.224264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.225064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.228689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] PATH_INFO: `/v3/projects/198def3b748243afa74128ce82e17d4c/users/c890d3ee5bb4433abc7572667a50e5ce/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.237479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1068/2136] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:34 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:34.239704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.239704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.239704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.255922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=198def3b748243afa74128ce82e17d4c, user_id=c890d3ee5bb4433abc7572667a50e5ce, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.259680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.279339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] MFA Rules not processed for user `75835548ad15448f80ed832e7aae5a38`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:34.281793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:34.352170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1069/2137] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:34.353530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.391251 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1069/2138] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:34 2022] PUT /identity/v3/projects/198def3b748243afa74128ce82e17d4c/users/c890d3ee5bb4433abc7572667a50e5ce/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:34.392572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.394557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:34.423812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c40a6-5653-402c-9e3c-3580fff5240a None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.462945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad67d8-a338-46bd-9722-5beee8f870ba admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.466878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.467502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.467780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.468054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.469862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:34.486349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.487129 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.487527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.487813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.488491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.490439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.529233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.534201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.541268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1070/2139] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:34.542405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.553329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f237022-5214-4130-984f-4dae1c0a2d01 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.558831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.559581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.559909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.560531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.561043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.562700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.573252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1071/2140] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:34.576978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.588307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab1d6-40a1-43fd-ac98-795ae446d069 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.595360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.596237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.596580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.597020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] PATH_INFO: `/v3/projects/198def3b748243afa74128ce82e17d4c/users/c890d3ee5bb4433abc7572667a50e5ce/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.600331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] RBAC: Authorizing `identity:create_grant(project_id=198def3b748243afa74128ce82e17d4c, user_id=c890d3ee5bb4433abc7572667a50e5ce, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.603248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.616390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1072/2141] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:34 2022] PUT /identity/v3/projects/198def3b748243afa74128ce82e17d4c/users/c890d3ee5bb4433abc7572667a50e5ce/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:34.662339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1070/2142] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:34 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:34.663668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.729316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e482fa-7aef-4378-ae8f-9ee808c8357f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.733853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.735075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.735075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.735075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.735628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.737263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.746881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1073/2143] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:34.751116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.761997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246a2ed-c785-4d5e-964d-b0a562568f69 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.771534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.771534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.771534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.771534 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.772013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.772013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014ade7-3a62-4f56-ba0b-142bc94d12b4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.789702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1071/2144] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:34.794666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:34.807117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90235-43be-46b6-8722-7fdea6240851 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:34.813021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:34.814392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.814755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.815151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] PATH_INFO: `/v3/projects/d3b7b2dcf4c24c5fab1fb0f790b8cc97/users/b00e0931dd52400b97dc052583fdb711/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.834157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b7b2dcf4c24c5fab1fb0f790b8cc97, user_id=b00e0931dd52400b97dc052583fdb711, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:34.841102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14670843-e454-48b9-988b-99e94b06b481 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:34.950256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.950256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.950256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.951189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1074/2145] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:34 2022] PUT /identity/v3/projects/d3b7b2dcf4c24c5fab1fb0f790b8cc97/users/b00e0931dd52400b97dc052583fdb711/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:34.963232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:34.963780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:34.964057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:34.987414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] MFA Rules not processed for user `ed5bde75f6d2477ab2747016ab5c309e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:34.988613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:35.027919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] MFA Rules not processed for user `b00e0931dd52400b97dc052583fdb711`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:35.029209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:35.097550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1072/2146] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:35.141029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1075/2147] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:35.153442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:35.155058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:35.238048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1edd3-617a-4972-9338-f50650d4f5c7 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:35.241726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:35.242341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:35.242619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:35.242923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:35.255294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:35.256806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c28d6b-10b0-4df1-9fdf-541763fade84 service glance] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:35.269320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:35.279303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1073/2148] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:35.329588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e007b-5b8d-4031-b091-5264e2815c28 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:35.333298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:35.333913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:35.334192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:35.334517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:35.347523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:35.347523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:35.357258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1076/2149] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:35.718343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:35.849341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:36.381608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abbc8f-4bb1-445b-816a-d1d290b7a91c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:36.382257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abbc8f-4bb1-445b-816a-d1d290b7a91c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:36.382957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5abbc8f-4bb1-445b-816a-d1d290b7a91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:36.420927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c5abbc8f-4bb1-445b-816a-d1d290b7a91c None None] MFA Rules not processed for user `c890d3ee5bb4433abc7572667a50e5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:36.422201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c5abbc8f-4bb1-445b-816a-d1d290b7a91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:36.503504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1074/2150] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:36.517734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:36.541807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034c14f-3e0c-4846-8415-139edfbf7dae service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:36.545749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:36.546360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:36.546826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:36.547105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:36.548933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:36.628942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:36.628942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:36.637207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1077/2151] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:38.237356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:38.264181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:38.757773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:38.855114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:38.946126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:38.991805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.020239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.266316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.461756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:39.462290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:39.462706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:39.463075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.490508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] MFA Rules not processed for user `ed5bde75f6d2477ab2747016ab5c309e`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:39.490836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:39.504327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1075/2152] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:39.546736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.775244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:39.906198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:40.209933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:40.346419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:40.751095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:40.951580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.454489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.483314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.641107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.675627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.841445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.871958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:41.984121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:42.012667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:42.180481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:42.276216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:42.564837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:42.564837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa86ce-ed09-4661-a83d-c27cff4d51e3 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:42.564837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:42.565473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:42.565574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:42.565832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:42.632044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:42.643849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:42.649432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:42.658407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1078/2153] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:56:43.191792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:43.193357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:43.205930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268fa2f-e8d6-402e-9493-e7d418879fc6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:43.209608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:43.210224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:43.213145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:43.213145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:43.220164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:43.221660 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc18ab-4aa1-43fe-82fb-e2748d47dce7 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:43.232011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1076/2154] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:43.260183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:43.328631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:43.643792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:44.031979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:44.304375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:44.381212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:44.383052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:44.449699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5dabb-cae2-49c8-9f14-9662ec4a06a1 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:44.454413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:44.455038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:44.455330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:44.455591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] PATH_INFO: `/v3/users/393f109857044c84920a91826ebadb9d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:44.456762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=393f109857044c84920a91826ebadb9d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:44.458712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:44.515826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:44.733051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393f109857044c84920a91826ebadb9d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:44.733802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393f109857044c84920a91826ebadb9d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:44.742474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393f109857044c84920a91826ebadb9d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:44.819858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393f109857044c84920a91826ebadb9d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:44.834139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393f109857044c84920a91826ebadb9d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:44.841326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1079/2155] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:44 2022] DELETE /identity/v3/users/393f109857044c84920a91826ebadb9d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:44.845604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:44.845889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:44.846180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:44.853967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:44.869697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:44.874057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:44.950786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1077/2156] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:44.957087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:44.958590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:45.021059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67807a-edc1-4ad1-bcf4-c32fea6e92cf admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:45.025503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:45.026949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:45.027303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:45.027744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:45.041866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:45.043370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6afb0a-99af-4577-aabb-0e697f5b5b43 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:45.053661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1080/2157] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:45.083247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:45.386379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:45.714404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:45.716155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:45.740314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d146648-5c38-486a-8df2-14daeb2d8de0 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:45.752736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:45.752736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:45.752736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:45.752736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] PATH_INFO: `/v3/projects/19de93aae58940babd61ffbf31f45c77` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:45.753082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19de93aae58940babd61ffbf31f45c77)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:45.755886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:45.939375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19de93aae58940babd61ffbf31f45c77', 'request_id': 'req-266bd473-0db4-4dc1-a315-629738000860'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:45.944781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19de93aae58940babd61ffbf31f45c77', 'request_id': 'req-266bd473-0db4-4dc1-a315-629738000860'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:45.952020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19de93aae58940babd61ffbf31f45c77', 'request_id': 'req-266bd473-0db4-4dc1-a315-629738000860'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:46.012670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] The token cache is being invalidate because project 19de93aae58940babd61ffbf31f45c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:46.012983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:46.013783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-266bd473-0db4-4dc1-a315-629738000860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:46.016100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1078/2158] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:45 2022] DELETE /identity/v3/projects/19de93aae58940babd61ffbf31f45c77 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:46.021738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.022045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.022390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.042386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:46.043589 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:46.110221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1081/2159] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:46.120411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.121005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.121005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.169220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:46.170550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:46.178967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1079/2160] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:46.184141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.194377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51cd1f-a944-4497-873f-48dfe1286766 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.199763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.200362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.200780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.201052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.201693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:46.201995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.203595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.216479 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1082/2161] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:46.221161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.231073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d4644-644b-4a06-b885-b931d28b172f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.235831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.236417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.236764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.237040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.237666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.239572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.406111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1080/2162] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:46 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:46.407432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.418132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702785-abd4-4777-82c1-29167e26ffd9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.476695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.477306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.477588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.477860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.478477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.480907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.563333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:46.637523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1083/2163] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:46 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:46.641405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.652499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf4d5-f745-461a-b58e-143225b263cf admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.657187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.657765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.658044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.658340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.659032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.660917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.671857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1081/2164] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:46.675943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.686113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c74dfd-396f-4187-81d4-2172163565f7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.690609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.691199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.691482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.691752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.692395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.694024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae4789-883e-4359-a386-f1168f136596 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.704843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1084/2165] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:46.708962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.720084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c43ce-8064-4444-8c4c-4d15b670d37f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:46.724627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:46.725301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.725589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.725851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] PATH_INFO: `/v3/projects/82f87452d1df46f9aa31a065265c39ed/users/1b62e09cda724c43b125f59924eed54f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.741324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f87452d1df46f9aa31a065265c39ed, user_id=1b62e09cda724c43b125f59924eed54f, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:46.744103 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:46.855218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1082/2166] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:46 2022] PUT /identity/v3/projects/82f87452d1df46f9aa31a065265c39ed/users/1b62e09cda724c43b125f59924eed54f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:46.866072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:46.866351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:46.866623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:46.879062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:46.899059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] MFA Rules not processed for user `1b62e09cda724c43b125f59924eed54f`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:46.900282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:46.981155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1085/2167] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:46.992174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:46.993941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:47.048624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:47.056438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb296b8-2113-4abf-83bc-72f828085054 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:47.060590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:47.061241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:47.061520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:47.061791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:47.070375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:47.071800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15571b3a-b0ce-4017-bb61-6cf5c51afd15 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:47.081441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1083/2168] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:47.106588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:47.138358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:47.199751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000b30e-6cf8-4829-88ed-d73f23d8d3c4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:47.205087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:47.205724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:47.205981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:47.206260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] PATH_INFO: `/v3/users/1b62e09cda724c43b125f59924eed54f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:47.213984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b62e09cda724c43b125f59924eed54f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:47.213984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:47.434174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b62e09cda724c43b125f59924eed54f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:47.434828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b62e09cda724c43b125f59924eed54f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:47.442270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b62e09cda724c43b125f59924eed54f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:47.454103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:47.518213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b62e09cda724c43b125f59924eed54f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:47.545723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b62e09cda724c43b125f59924eed54f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:47.558283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1086/2169] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:47 2022] DELETE /identity/v3/users/1b62e09cda724c43b125f59924eed54f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:47.560742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:47.561136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:47.561514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:47.599937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:47.601523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:47.669226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1084/2170] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:47.679351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:47.680815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:47.723240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:47.746389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb77a4-e639-4822-934f-7405a83dd29b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:47.750096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:47.750706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:47.750984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:47.751257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:47.761334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:47.762853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357dcfa-a6d6-4664-aaa5-95c73a71b0b8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:47.773305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1087/2171] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:48.007405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:48.193068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:48.267214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:48.267214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:48.295943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae5c12-6bc1-41b7-bef3-3a71c58451ad None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:48.295943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:48.295943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:48.295943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:48.296848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] PATH_INFO: `/v3/projects/82f87452d1df46f9aa31a065265c39ed` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:48.296848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f87452d1df46f9aa31a065265c39ed)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:48.297985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:48.470097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f87452d1df46f9aa31a065265c39ed', 'request_id': 'req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:48.477302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f87452d1df46f9aa31a065265c39ed', 'request_id': 'req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:48.486599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f87452d1df46f9aa31a065265c39ed', 'request_id': 'req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:48.580703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] The token cache is being invalidate because project 82f87452d1df46f9aa31a065265c39ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:48.581001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:48.581663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-9aeb4e73-b8c7-473c-9d4f-1b0b1e2bf36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:48.583240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1085/2172] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:48 2022] DELETE /identity/v3/projects/82f87452d1df46f9aa31a065265c39ed => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:48.588868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91de07-3e82-4457-accf-527e185627e8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:48.589372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91de07-3e82-4457-accf-527e185627e8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:48.589454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91de07-3e82-4457-accf-527e185627e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:48.630259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-db91de07-3e82-4457-accf-527e185627e8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:48.630259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-db91de07-3e82-4457-accf-527e185627e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:48.739962 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1088/2173] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:48.747403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:48.747687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:48.747958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:48.773272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:48.774749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:48.799808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1086/2174] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:48.804005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91de07-3e82-4457-accf-527e185627e8 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:48.819593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91de07-3e82-4457-accf-527e185627e8 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:48.824643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:48.825032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:48.825311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:48.825585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:48.826475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:48.826599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:48.828166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:48.843071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1089/2175] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:48.847623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:48.872314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbcaf8-2a3e-4dc8-95b6-3d53a5e91da9 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:48.880346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:48.881301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:48.881697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:48.882057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:48.883553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:48.886551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.005011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1087/2176] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:48 2022] POST /identity/v3/projects => generated 399 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:49.014374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:49.037000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1a2f7-a774-44db-a8e2-f3b3ffc76727 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:49.119806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:49.120401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.120766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.121112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.121772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:49.129369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.303586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1090/2177] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:49 2022] POST /identity/v3/users => generated 300 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:49.308718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:49.320085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:49.326425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d9ac0-66cd-4c18-af26-7098f41f7f17 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:49.333989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c261498-4243-472d-8616-140843685741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:49.335378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261498-4243-472d-8616-140843685741 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.336386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261498-4243-472d-8616-140843685741 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.337128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c261498-4243-472d-8616-140843685741 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.338912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c261498-4243-472d-8616-140843685741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:49.341865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c261498-4243-472d-8616-140843685741 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.360911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1088/2178] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:49.364425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:49.393252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d4fe-4764-448d-a7e1-c0995333b9cd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:49.407687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:49.408257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.408538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.408977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.409545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:49.411457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92691518-2741-4063-8b54-bc98d4678bd9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.426879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1091/2179] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:49.431384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c261498-4243-472d-8616-140843685741 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:49.446449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c261498-4243-472d-8616-140843685741 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:49.454183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:49.460812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.460812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.460812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] PATH_INFO: `/v3/projects/ce67f3fbabff4036bdfce7fca25ba562/users/fa7add067ac74fe5a6dd7626ecf5c0be/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.473401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce67f3fbabff4036bdfce7fca25ba562, user_id=fa7add067ac74fe5a6dd7626ecf5c0be, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:49.476191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.599149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1089/2180] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:49 2022] PUT /identity/v3/projects/ce67f3fbabff4036bdfce7fca25ba562/users/fa7add067ac74fe5a6dd7626ecf5c0be/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:49.611217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:49.612405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.612996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.613153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.649375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] MFA Rules not processed for user `fa7add067ac74fe5a6dd7626ecf5c0be`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:49.649375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:49.740987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1092/2181] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:49 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:49.759973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:49.765461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:49.830228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3f749-c69a-4669-8807-af99ddea10d9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:49.834382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:49.834998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:49.835277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:49.835548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:49.851186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:49.852835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:49.866989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1090/2182] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:50.813547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:51.282917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:52.391441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:52.720948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:53.048404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:53.218566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:53.305783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:53.729783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:53.732352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:53.804070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc2b6b-b737-440c-9e31-0ac154b04795 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:53.809273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:53.809901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:53.810178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:53.810450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] PATH_INFO: `/v3/users/c772000142f14498933a685d3855bc02` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:53.811568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c772000142f14498933a685d3855bc02)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:53.813741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:53.972058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:54.087482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c772000142f14498933a685d3855bc02'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.087482 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c772000142f14498933a685d3855bc02'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.090936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c772000142f14498933a685d3855bc02'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.209019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c772000142f14498933a685d3855bc02'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.227766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b6402214-83b2-474a-a5db-dd7235df71bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c772000142f14498933a685d3855bc02'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.238637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1093/2183] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:56:53 2022] DELETE /identity/v3/users/c772000142f14498933a685d3855bc02 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:54.277376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:54.522462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:54.533973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56042c01-f70c-4cf5-85f7-dfb6a7aa55e6 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:54.597053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:54.597698 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:54.597977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:54.598251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] PATH_INFO: `/v3/projects/601816703e524795ab40b1dea9393870` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:54.599393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] RBAC: Authorizing `identity:delete_project(project_id=601816703e524795ab40b1dea9393870)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:54.601342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:54.726847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601816703e524795ab40b1dea9393870', 'request_id': 'req-e92dc2ad-15dc-4678-98e8-3b3771c9986e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.735567 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601816703e524795ab40b1dea9393870', 'request_id': 'req-e92dc2ad-15dc-4678-98e8-3b3771c9986e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.748677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601816703e524795ab40b1dea9393870', 'request_id': 'req-e92dc2ad-15dc-4678-98e8-3b3771c9986e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.834116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] The token cache is being invalidate because project 601816703e524795ab40b1dea9393870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:56:54.834414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.835416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e92dc2ad-15dc-4678-98e8-3b3771c9986e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:56:54.837333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1091/2184] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:56:54 2022] DELETE /identity/v3/projects/601816703e524795ab40b1dea9393870 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:54.842178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:54.842644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:54.842724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:54.862923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:54.864134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:54.932221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1094/2185] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:56:54.940796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:54.941083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:54.941360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:54.961730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:54.963309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:54.971978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1092/2186] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:54.979224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:54.989179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffe064-6828-46e4-b29a-90958de04273 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:54.994301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:54.994896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:54.995177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:54.995451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:54.996100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:56:54.996404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:54.998046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.010668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1095/2187] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:56:55.014877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.024367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebc654-e1a9-48c8-ba63-b90a1422c0b8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.029253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.029870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.030147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.030413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.031037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.032975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.160506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1093/2188] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:55 2022] POST /identity/v3/projects => generated 369 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:56:55.165057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.177855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604ab74-ba00-4a37-9937-b2d4271fa7d3 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.232290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:55.248266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.248905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.249188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.249461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.250105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.252217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.358811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1096/2189] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:55 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:55.363185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.377257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116025d-11b7-4b30-8747-91a1b0e4cedb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.381153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:55.385725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.386318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.386589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.386860 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.387757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.389652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.412740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1094/2190] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:56:55.420749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.432702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fa49a-4760-4d2e-bbd9-0a0689484aae admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.440261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.440979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.441276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.443113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.443113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.447631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38a49c-75bd-4214-aa87-1534132b9277 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.457971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1097/2191] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:55.468420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.479837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e145-f391-46f8-ba5f-ee29a56d4793 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.487852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.489694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.489694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.489694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] PATH_INFO: `/v3/projects/fdacd677c08040d58e81c8000607d909/users/1693da72f2374aa29520e6b99a0ac26b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.509525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdacd677c08040d58e81c8000607d909, user_id=1693da72f2374aa29520e6b99a0ac26b, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.512785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:55.659152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1095/2192] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:55 2022] PUT /identity/v3/projects/fdacd677c08040d58e81c8000607d909/users/1693da72f2374aa29520e6b99a0ac26b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:56:55.670976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.671804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.672156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.707052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] MFA Rules not processed for user `1693da72f2374aa29520e6b99a0ac26b`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:55.708165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:55.809459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1098/2193] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:55.814157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:55.898357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1574e0-0403-473f-aeaa-e3b683d8e0aa admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:55.904071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:55.904634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:55.904977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:55.905249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:55.906067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:55.908146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.015007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1096/2194] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:56:55 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:56.015744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.028229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3c72a-6d5c-4cd9-b847-f0f0bef21690 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.108629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.110095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.110512 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.110877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.112582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.114664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.247226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:56.254538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1099/2195] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:56:56 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:56:56.255895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.266084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3c90d-b88f-4680-8fcc-d7cc65d45de0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.270522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.271110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.271383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.271649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.277521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.277521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.284378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1097/2196] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:56.288941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.299522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa24af-997e-44b0-9692-d4f33027b142 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.304209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.304895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.305179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.305453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] PATH_INFO: `/v3/projects/e913c73ec03b406dbb34fc1bd5546034/users/09ead671782144fb8a1d4ebf2e2e30fc/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.321442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e913c73ec03b406dbb34fc1bd5546034, user_id=09ead671782144fb8a1d4ebf2e2e30fc, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.324174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.413072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1100/2197] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:56 2022] PUT /identity/v3/projects/e913c73ec03b406dbb34fc1bd5546034/users/09ead671782144fb8a1d4ebf2e2e30fc/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:56.417696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.429583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e683b3-dad2-4cdc-89c4-aba0b08d57ca admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.486349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.486957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.487235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.487508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.488158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.489769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c718-ed86-4b39-a140-42376ef07034 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.502953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1098/2198] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:56:56.504126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.514586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457d2dc-fedf-4d39-a49e-ecd842512d72 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.519215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.519865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.520142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.520412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] PATH_INFO: `/v3/projects/e913c73ec03b406dbb34fc1bd5546034/users/09ead671782144fb8a1d4ebf2e2e30fc/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.522334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e913c73ec03b406dbb34fc1bd5546034, user_id=09ead671782144fb8a1d4ebf2e2e30fc, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.526296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.535978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1101/2199] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:56:56 2022] PUT /identity/v3/projects/e913c73ec03b406dbb34fc1bd5546034/users/09ead671782144fb8a1d4ebf2e2e30fc/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:56:56.547904 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.548195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.548476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.575701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] MFA Rules not processed for user `09ead671782144fb8a1d4ebf2e2e30fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:56:56.576949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:56.650652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1099/2200] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:56:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:56:56.663555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:56.665290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:56.665419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:56:56.725986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549ac48-35f7-47b1-a327-a98e1c74080b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:56.729909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:56.730525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:56.730807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:56.731080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:56.741133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:56.742573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:56.753453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1102/2201] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:56.949822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:57.096694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:56:57.107985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5c744-7c5a-4cd6-af21-ade139e7d7cd None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:56:57.111653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:56:57.112259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:56:57.115350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:56:57.115350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:56:57.174278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:56:57.175644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28797c63-e1ab-4605-8400-e8a88803204b service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:56:57.186420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1100/2202] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:56:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:56:57.905481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.100622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.225207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.303195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.390799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.452032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:56:58.566476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:07.968258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:08.647817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:08.723033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:08.725127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:08.801522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a01d8-20bb-405b-8cc1-7910545ee54c service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:08.807429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:08.808244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:08.808706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:08.809138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] PATH_INFO: `/v3/users/fa7add067ac74fe5a6dd7626ecf5c0be` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:08.810559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa7add067ac74fe5a6dd7626ecf5c0be)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:08.812740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:09.023649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa7add067ac74fe5a6dd7626ecf5c0be'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:09.024433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa7add067ac74fe5a6dd7626ecf5c0be'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:09.032478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa7add067ac74fe5a6dd7626ecf5c0be'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:09.121951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa7add067ac74fe5a6dd7626ecf5c0be'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:09.137747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa7add067ac74fe5a6dd7626ecf5c0be'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:09.149562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1103/2203] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:08 2022] DELETE /identity/v3/users/fa7add067ac74fe5a6dd7626ecf5c0be => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:09.151458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:09.151946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:09.152035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:09.178766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:09.180336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:09.255900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1101/2204] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:09.266944 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:09.268632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:09.348928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb612d0c-d878-4333-8c53-51fdf60e3ffc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:09.354019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:09.354653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:09.354930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:09.356780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:09.365612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:09.367286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:09.398772 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1104/2205] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:09.986118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:09.989222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:10.011662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3e367-3aea-4784-ba15-00362135fb94 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:10.018238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:10.018238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:10.018238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:10.018238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] PATH_INFO: `/v3/projects/ce67f3fbabff4036bdfce7fca25ba562` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:10.019372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce67f3fbabff4036bdfce7fca25ba562)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:10.021335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:10.155099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce67f3fbabff4036bdfce7fca25ba562', 'request_id': 'req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:10.161056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce67f3fbabff4036bdfce7fca25ba562', 'request_id': 'req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:10.168094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce67f3fbabff4036bdfce7fca25ba562', 'request_id': 'req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:10.299150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] The token cache is being invalidate because project ce67f3fbabff4036bdfce7fca25ba562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:10.299150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:10.299150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:10.299150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1102/2206] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:09 2022] DELETE /identity/v3/projects/ce67f3fbabff4036bdfce7fca25ba562 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:14.678682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:14.683930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:14.704880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac305c6-560a-47dc-8c99-93e296a4ed41 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:14.730443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:14.789276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:14.790263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:14.790473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:14.790742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] PATH_INFO: `/v3/users/379fe93540ee4318be1b10dbcc417148` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:14.791931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=379fe93540ee4318be1b10dbcc417148)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:14.794175 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:15.046062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '379fe93540ee4318be1b10dbcc417148'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.046709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '379fe93540ee4318be1b10dbcc417148'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.053798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '379fe93540ee4318be1b10dbcc417148'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.144522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '379fe93540ee4318be1b10dbcc417148'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.157165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '379fe93540ee4318be1b10dbcc417148'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.163445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1105/2207] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:14 2022] DELETE /identity/v3/users/379fe93540ee4318be1b10dbcc417148 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:15.420099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:15.421735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:15.436182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda60a74-f326-4dc0-8ecc-d4e7270ab8bb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:15.499090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:15.499768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:15.500053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:15.500329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] PATH_INFO: `/v3/projects/0b4e12b2521a40ae8aecd5c818bfc623` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:15.501499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4e12b2521a40ae8aecd5c818bfc623)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:15.503457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:15.738938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:15.764970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4e12b2521a40ae8aecd5c818bfc623', 'request_id': 'req-8aabef0c-1901-40a0-b62c-c6a7139f0465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.771344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4e12b2521a40ae8aecd5c818bfc623', 'request_id': 'req-8aabef0c-1901-40a0-b62c-c6a7139f0465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.778925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4e12b2521a40ae8aecd5c818bfc623', 'request_id': 'req-8aabef0c-1901-40a0-b62c-c6a7139f0465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.920058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] The token cache is being invalidate because project 0b4e12b2521a40ae8aecd5c818bfc623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:15.920352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.921000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:15.922538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1103/2208] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:15 2022] DELETE /identity/v3/projects/0b4e12b2521a40ae8aecd5c818bfc623 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:15.926596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:15.928438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:15.943775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085dd74-8f93-4495-8c23-dd29ddb88580 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:16.014952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:16.015655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:16.015941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:16.016216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] PATH_INFO: `/v3/users/c890d3ee5bb4433abc7572667a50e5ce` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:16.017659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c890d3ee5bb4433abc7572667a50e5ce)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:16.019745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:16.272567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:16.275320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c890d3ee5bb4433abc7572667a50e5ce'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:16.276033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c890d3ee5bb4433abc7572667a50e5ce'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:16.282951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c890d3ee5bb4433abc7572667a50e5ce'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:16.367419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c890d3ee5bb4433abc7572667a50e5ce'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:16.380160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-160ec358-b92c-4757-a5e5-1600bf3388cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c890d3ee5bb4433abc7572667a50e5ce'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:16.387124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1106/2209] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:15 2022] DELETE /identity/v3/users/c890d3ee5bb4433abc7572667a50e5ce => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:16.706285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:16.707799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:16.722984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabef0c-1901-40a0-b62c-c6a7139f0465 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:16.861634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00291bee-ee93-41b7-aca2-5bf56f30dcd2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:16.861922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00291bee-ee93-41b7-aca2-5bf56f30dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:16.862197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00291bee-ee93-41b7-aca2-5bf56f30dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:16.863675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:16.864416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:16.864864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:16.865279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] PATH_INFO: `/v3/projects/198def3b748243afa74128ce82e17d4c` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:16.866890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=198def3b748243afa74128ce82e17d4c)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:16.869821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:16.885947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-00291bee-ee93-41b7-aca2-5bf56f30dcd2 None None] MFA Rules not processed for user `ed5bde75f6d2477ab2747016ab5c309e`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:16.887106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-00291bee-ee93-41b7-aca2-5bf56f30dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:16.950628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1107/2210] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:17.016176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198def3b748243afa74128ce82e17d4c', 'request_id': 'req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.023143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198def3b748243afa74128ce82e17d4c', 'request_id': 'req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.029892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198def3b748243afa74128ce82e17d4c', 'request_id': 'req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.166966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] The token cache is being invalidate because project 198def3b748243afa74128ce82e17d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:17.167292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.169322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8eb50905-4a09-4894-9cb6-0b9b2d2e63f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.170898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1104/2211] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:16 2022] DELETE /identity/v3/projects/198def3b748243afa74128ce82e17d4c => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:17.176220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.176778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.177211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.200801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:17.202037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.268950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1108/2212] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:17.277747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.278041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.278371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.299427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:17.300679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.310058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1105/2213] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:17.314092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.324622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efc417-162b-40ab-b8ed-8c6d81692bc8 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.330102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.330607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.330890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.331163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.331917 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:17.332222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.333947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.346121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.347714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.349386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1109/2214] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:17.354407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.363889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7d399-5624-4f9a-b386-07bbfe3f8ba3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.364632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377e01a-16d2-4788-8dde-cb6975d2fae9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.369020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.369657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.369799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.370012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.370278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] PATH_INFO: `/v3/users/ceb09f6e4b1e48c2a84096c36cdb2c2e` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.370393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.370601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.370882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.371484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.371587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb09f6e4b1e48c2a84096c36cdb2c2e)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.373454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.373579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.502625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1110/2215] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:17.504275 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.544360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:17.575364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b37f2-b6fb-498e-ac21-3504c21d9b14 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.580296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.580949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.581230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.581490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.582130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.584094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.616730 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:17.645583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb09f6e4b1e48c2a84096c36cdb2c2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.656778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb09f6e4b1e48c2a84096c36cdb2c2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.663403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb09f6e4b1e48c2a84096c36cdb2c2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.663403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb09f6e4b1e48c2a84096c36cdb2c2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.716607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1111/2216] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:17.718544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.720583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.732607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e22145-d48e-4a2d-9112-470f33db9ebf admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.737838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.738435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.738711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.738985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.739628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.741326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.742394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a2b1c586-b47d-4283-85ee-a185a09a16aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb09f6e4b1e48c2a84096c36cdb2c2e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:17.755469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1112/2217] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:17.759211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.763608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1106/2218] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:17 2022] DELETE /identity/v3/users/ceb09f6e4b1e48c2a84096c36cdb2c2e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:17.768079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158315-8df9-4438-8001-b180c1ce186c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.768579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158315-8df9-4438-8001-b180c1ce186c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.768966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158315-8df9-4438-8001-b180c1ce186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.772079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5eae8f-d1fe-4d71-b915-58e69405d775 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.792196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-81158315-8df9-4438-8001-b180c1ce186c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:17.793545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-81158315-8df9-4438-8001-b180c1ce186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.839764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.840378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.840676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.840964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.841610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.843243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.855869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1113/2219] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:17.860337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.872911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1107/2220] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:17.873140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd981580-94a4-4184-a868-3160f82fd3a0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.876596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.877271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.877652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.877811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] PATH_INFO: `/v3/projects/4fbdbb1929714d309a40130cb7414d43/users/520f62850bc248b9b00139d92821642a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.881118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81158315-8df9-4438-8001-b180c1ce186c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:17.883995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-81158315-8df9-4438-8001-b180c1ce186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:17.896201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fbdbb1929714d309a40130cb7414d43, user_id=520f62850bc248b9b00139d92821642a, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.900124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9c380-8a1d-4052-bf26-70c0569e0614 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.953953 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81158315-8df9-4438-8001-b180c1ce186c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:17.959024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:17.959670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.960152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.960264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.974066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:17.975130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1114/2221] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:17 2022] PUT /identity/v3/projects/4fbdbb1929714d309a40130cb7414d43/users/520f62850bc248b9b00139d92821642a/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:17.985044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:17.986804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:17.987098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:17.987372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:17.987855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1108/2222] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:18.035662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] MFA Rules not processed for user `520f62850bc248b9b00139d92821642a`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:18.037965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:18.124042 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1115/2223] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:18.136149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:18.137879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:18.201325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b58ecf-cdd2-45b7-a1f1-d2011f04f9bd service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:18.206489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:18.280721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:18.281288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:18.281583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:18.281838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:18.292979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:18.294714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:18.319600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1109/2224] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:18.483576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:18.485204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:18.499242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:18.551322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84439d-51d2-49c9-977e-7c1cf6b2cb42 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:18.556064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:18.556718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:18.556996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:18.557261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] PATH_INFO: `/v3/projects/033ea0bc60d544bf9a7ba8abbafafa0f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:18.558391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=033ea0bc60d544bf9a7ba8abbafafa0f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:18.560283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:18.693584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033ea0bc60d544bf9a7ba8abbafafa0f', 'request_id': 'req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:18.807453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033ea0bc60d544bf9a7ba8abbafafa0f', 'request_id': 'req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:18.815645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033ea0bc60d544bf9a7ba8abbafafa0f', 'request_id': 'req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:18.827708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] The token cache is being invalidate because project 033ea0bc60d544bf9a7ba8abbafafa0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:18.827998 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:18.828679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:18.830651 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1116/2225] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:18 2022] DELETE /identity/v3/projects/033ea0bc60d544bf9a7ba8abbafafa0f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:18.834334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:18.845120 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed670e15-2443-4386-bb1d-0e939bc05670 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:18.902833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:18.903480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:18.903740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:18.904012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] PATH_INFO: `/v3/users/75835548ad15448f80ed832e7aae5a38` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:18.905156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] RBAC: Authorizing `identity:delete_user(user_id=75835548ad15448f80ed832e7aae5a38)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:18.907070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:19.143149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75835548ad15448f80ed832e7aae5a38'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:19.150393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75835548ad15448f80ed832e7aae5a38'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:19.151086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75835548ad15448f80ed832e7aae5a38'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:19.151729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75835548ad15448f80ed832e7aae5a38'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:19.275711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75835548ad15448f80ed832e7aae5a38'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:19.297315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1110/2226] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:18 2022] DELETE /identity/v3/users/75835548ad15448f80ed832e7aae5a38 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:19.806277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:19.808184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:19.821894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1faca-a79e-4d8d-a8a7-407b0f21fbcd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:19.886100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:19.886835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:19.887116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:19.887383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] PATH_INFO: `/v3/projects/09dac59775044c43b70b09d9241cbe9e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:19.888629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=09dac59775044c43b70b09d9241cbe9e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:19.890689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:19.986556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:19.990224 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09dac59775044c43b70b09d9241cbe9e', 'request_id': 'req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.029537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.030981 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.043906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a417f-a566-4377-9750-37a76b1786af admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.058315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09dac59775044c43b70b09d9241cbe9e', 'request_id': 'req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.065356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09dac59775044c43b70b09d9241cbe9e', 'request_id': 'req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.077045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] The token cache is being invalidate because project 09dac59775044c43b70b09d9241cbe9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:20.077422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.078192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cfa783f6-da29-41d9-84d7-1ebd17f6c7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.079032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:20.080516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1117/2227] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:19 2022] DELETE /identity/v3/projects/09dac59775044c43b70b09d9241cbe9e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:20.094195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334de43-05cf-408a-b4fe-9d214333aef5 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.094622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334de43-05cf-408a-b4fe-9d214333aef5 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.095004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334de43-05cf-408a-b4fe-9d214333aef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.123894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.124447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5334de43-05cf-408a-b4fe-9d214333aef5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:20.125280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.125506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.125749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5334de43-05cf-408a-b4fe-9d214333aef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.131583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] PATH_INFO: `/v3/users/b00e0931dd52400b97dc052583fdb711` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.131583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00e0931dd52400b97dc052583fdb711)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.131583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.144031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1118/2228] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:20.153263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.158136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.158136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.176515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:20.178576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.204833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1119/2229] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:20.209775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.215697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34443d59-7afc-4880-b6ce-213f8d775d35 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.222183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.222659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.223161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.223459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.224127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:20.224430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.226331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.239433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1120/2230] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:20.244687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.254433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e6cd1-c8b0-429a-bd67-77ab76577a9b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.260167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.260817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.261097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.261372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.262441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.263926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.352901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00e0931dd52400b97dc052583fdb711'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.360992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00e0931dd52400b97dc052583fdb711'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.361690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00e0931dd52400b97dc052583fdb711'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.362381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00e0931dd52400b97dc052583fdb711'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.407638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1121/2231] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:20.408129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.410441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.478513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19926-f274-4fbb-be4b-affbe0045ec7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.481174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-90238218-05f3-4e72-82c1-f1ff5bdf25a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00e0931dd52400b97dc052583fdb711'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:20.483478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.484092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.484367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.484752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.485504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.488951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.513776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1111/2232] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:20 2022] DELETE /identity/v3/users/b00e0931dd52400b97dc052583fdb711 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:20.557305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.557305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.557305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.557305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:20.557305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.623283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1122/2233] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:20.629896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.635247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1112/2234] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:20.644457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.645188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a5a15-c88d-4af1-9a62-7a82cd34fcb9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.646769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:20.650801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.651747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.654983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.654983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.654983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.655510 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.668739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1123/2235] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:57:20.673110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.686320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b523c6-500f-43fb-98ee-4d9f2fa3ab92 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.693022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.693963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.694392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.694977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.696391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.698655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.719839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1124/2236] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:20.724888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488387d-ed7e-4dd3-8c0c-f06177c40289 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.730761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.733175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.733557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.733978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:20.734401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.752109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c0f01-18ee-4ce2-a9fb-0f3007859bec admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:20.759026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.762331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816332e3-5306-41d6-b8fb-452a78d3a853 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.764544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:20.765241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.765518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.765792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] PATH_INFO: `/v3/projects/12de18c1662c45e38619c52ba37e4744/users/ac109b86fb214c0da757b3808b73dede/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.786653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1113/2237] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:20.794533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12de18c1662c45e38619c52ba37e4744, user_id=ac109b86fb214c0da757b3808b73dede, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:20.801459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:20.893898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1125/2238] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:20 2022] PUT /identity/v3/projects/12de18c1662c45e38619c52ba37e4744/users/ac109b86fb214c0da757b3808b73dede/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:20.905766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:20.906085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:20.906353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:20.935793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] MFA Rules not processed for user `ac109b86fb214c0da757b3808b73dede`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:20.937029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.023438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1114/2239] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:21.028273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.071246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:21.074685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:21.125434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b4392-d643-44d1-85d3-8ebd231d5ae9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.130590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.131214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.132061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.132061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.132376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.134457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.246173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1126/2240] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:21 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:21.250194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.261457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7b923-2c4c-444d-9402-22658fd18207 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.323861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.324482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.324792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.325070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.325749 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.327801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.361189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.363064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.383260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b92352-73f3-4537-9a9e-5ea25db47916 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.388595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.389265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.389548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.389869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] PATH_INFO: `/v3/projects/d3b7b2dcf4c24c5fab1fb0f790b8cc97` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.390961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3b7b2dcf4c24c5fab1fb0f790b8cc97)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.392953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.419704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1115/2241] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:21 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:21.423761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.465224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8eeac-363b-437d-94b9-8cc79c577f38 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.477035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.477721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.478007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.478284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.480189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.481954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.494400 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1116/2242] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:57:21.498799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.513424 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944959b-c31e-4188-aaca-8b4a04c08f80 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.518920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.521444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.521754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.522038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] PATH_INFO: `/v3/projects/e95f4972414743c7ad09c28667b61260/users/74809413f5da4de298f66d54d8a07a02/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.540108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95f4972414743c7ad09c28667b61260, user_id=74809413f5da4de298f66d54d8a07a02, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.542750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.667631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1117/2243] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:21 2022] PUT /identity/v3/projects/e95f4972414743c7ad09c28667b61260/users/74809413f5da4de298f66d54d8a07a02/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:21.672216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.678095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3b7b2dcf4c24c5fab1fb0f790b8cc97', 'request_id': 'req-fe69ebec-ecc7-4605-915f-224e9b579278'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:21.762244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad5c2-1daf-46c0-a2b9-9df13bccb359 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.767070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.767690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.767956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.768228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.772272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.772272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.781903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1118/2244] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:57:21.790031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.809099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d64aa3-9c9b-421c-8615-b1fc96c737bf admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:21.819322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:21.820024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.820903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.821197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] PATH_INFO: `/v3/projects/e95f4972414743c7ad09c28667b61260/users/74809413f5da4de298f66d54d8a07a02/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.823281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95f4972414743c7ad09c28667b61260, user_id=74809413f5da4de298f66d54d8a07a02, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:21.824640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3b7b2dcf4c24c5fab1fb0f790b8cc97', 'request_id': 'req-fe69ebec-ecc7-4605-915f-224e9b579278'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:21.826118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf17b-87aa-479f-ad86-87617329f3b5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:21.832339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3b7b2dcf4c24c5fab1fb0f790b8cc97', 'request_id': 'req-fe69ebec-ecc7-4605-915f-224e9b579278'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:21.838561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1119/2245] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:21 2022] PUT /identity/v3/projects/e95f4972414743c7ad09c28667b61260/users/74809413f5da4de298f66d54d8a07a02/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:21.850122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] The token cache is being invalidate because project d3b7b2dcf4c24c5fab1fb0f790b8cc97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:21.850418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:21.851782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-fe69ebec-ecc7-4605-915f-224e9b579278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:21.852832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.853336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.853437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1127/2246] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:21 2022] DELETE /identity/v3/projects/d3b7b2dcf4c24c5fab1fb0f790b8cc97 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:21.857255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.863912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e979be-f5c0-49a7-a50b-57e067cf9fb6 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.863912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e979be-f5c0-49a7-a50b-57e067cf9fb6 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.863912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e979be-f5c0-49a7-a50b-57e067cf9fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.890123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-72e979be-f5c0-49a7-a50b-57e067cf9fb6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:21.891519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-72e979be-f5c0-49a7-a50b-57e067cf9fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.895298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] MFA Rules not processed for user `74809413f5da4de298f66d54d8a07a02`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:21.899324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.965413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1128/2247] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:21.973227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:21.973509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:21.973775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:21.981670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1120/2248] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:21.989401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:21.990821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:21.998216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:21.999108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.007511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1129/2249] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:22.012310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.022797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d504a1-5c4d-4eac-b2e3-14776ce50f93 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.030023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.030023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.030023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.030023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.030365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:22.030365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.031956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.047904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1130/2250] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:57:22.048806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.060563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe0a9d-b5e0-4189-9cdb-f9349de3feea admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.061545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ff742-2ca0-41e2-916b-34fefae3c210 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.065005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.065695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.065923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.066190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.066530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.067243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.067612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.068154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.068720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.071361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.076195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.077839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.087586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1121/2251] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:22.116693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.118729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.132363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116ddfe-7c5e-4959-838e-a8fe746ea3f9 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.137624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.138298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.138571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.138838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] PATH_INFO: `/v3/users/ac109b86fb214c0da757b3808b73dede` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.140063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac109b86fb214c0da757b3808b73dede)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.142137 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.191560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1131/2252] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:22 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:22.193396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.287480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269257e9-17e3-42ea-ad35-268ddb09cb23 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.293155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.293774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.294058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.294380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.295408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.298586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.339017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac109b86fb214c0da757b3808b73dede'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:22.347827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac109b86fb214c0da757b3808b73dede'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:22.348585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac109b86fb214c0da757b3808b73dede'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:22.349269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac109b86fb214c0da757b3808b73dede'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:22.440398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1132/2253] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:22 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:22.445120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.445996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-17f342a9-2f82-47d6-a214-76661cf86a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac109b86fb214c0da757b3808b73dede'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:22.448429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.460196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbd1dc-0c51-41b3-8c25-19eecd2c8dd0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.465322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.465933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.466209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.466480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.467128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.468897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.473557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1122/2254] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:22 2022] DELETE /identity/v3/users/ac109b86fb214c0da757b3808b73dede => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:22.475890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.479837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.479837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.480499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1133/2255] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:57:22.486588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.497765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42ab3d-6651-4b5e-ba6d-bea95c1e980a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.500844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:22.502280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.568983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.569614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.570010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.570315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.571171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.572567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.578468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1123/2256] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:22.583227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1134/2257] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:22.587741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.589254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.592385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.600191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f3d6a-0bd5-47b6-93e7-6b6607431543 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.605563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.606831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.607142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.607423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] PATH_INFO: `/v3/projects/33de9ced59b54846a2d5b81a1c7f7df0/users/fc7c2f4fe9d54f7ab2debc508e4d4153/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.626681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=33de9ced59b54846a2d5b81a1c7f7df0, user_id=fc7c2f4fe9d54f7ab2debc508e4d4153, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.630405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.637388 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:22.666923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac01f6-f348-4566-82be-21089e934bfa admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.674579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.676272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.676559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.676896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.687863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.689672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe093ec7-65ee-4958-b2c0-561245b45393 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.702097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1135/2258] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:22.739315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:22.759134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1124/2259] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:22 2022] PUT /identity/v3/projects/33de9ced59b54846a2d5b81a1c7f7df0/users/fc7c2f4fe9d54f7ab2debc508e4d4153/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:22.761212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.761562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.761838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.787204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.790744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.813384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] MFA Rules not processed for user `fc7c2f4fe9d54f7ab2debc508e4d4153`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:22.814601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:22.905160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7d9f6-cb1e-42a7-a4c9-428900b2b1df admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.905639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1136/2260] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:22.910170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:22.911248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.914602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.914602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.914602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] PATH_INFO: `/v3/users/520f62850bc248b9b00139d92821642a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.915093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=520f62850bc248b9b00139d92821642a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.917082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:22.923607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8da80-2057-4758-a96c-8967eb0f452d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:22.928429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:22.929151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:22.929517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:22.929849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:22.930587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:22.945844 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.061167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1137/2261] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:22 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:23.065248 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.123901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '520f62850bc248b9b00139d92821642a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.132300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '520f62850bc248b9b00139d92821642a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.133629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '520f62850bc248b9b00139d92821642a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.134249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '520f62850bc248b9b00139d92821642a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.148169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b7098-935c-4cda-94eb-5364ce44ca64 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.153500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.154131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.154409 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.154682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.155326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.157316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.216398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-8a177f4e-5fe8-455b-850c-39b6fadb60c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '520f62850bc248b9b00139d92821642a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.236144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1125/2262] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:22 2022] DELETE /identity/v3/users/520f62850bc248b9b00139d92821642a => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:23.240805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.241153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.241512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.261041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1138/2263] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:23 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:23.262465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.264210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.271004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:23.272593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.279015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982609f2-3ced-4993-be4e-a7925bcafd4b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.406158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.406798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.407186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.407347 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.408010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.411136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.420498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1126/2264] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:23.427385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.429234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.429377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1139/2265] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:23.434471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.451533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2814c-60bb-4163-ab44-6b9d291ec0a0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.457306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.458052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.458402 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.459134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.460516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.463074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.476673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1140/2266] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:23.481087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.492689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55da5af-f136-4065-8cd0-5b39c0c74630 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.497955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.498604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.498886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.499159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] PATH_INFO: `/v3/projects/e5f6acc6ed434c65964a5cdb81ae5e04/users/d77fe8d3046e4cfdb5b3f5bc15b31436/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.509484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa2494-d1c9-4156-af60-230d82adbc43 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.514620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.515387 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.515723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.516000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.523557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f6acc6ed434c65964a5cdb81ae5e04, user_id=d77fe8d3046e4cfdb5b3f5bc15b31436, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.526414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.526634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e5e6-4214-4c6c-8bc1-2bd1c214f491 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.527985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.539095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1127/2267] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:23.647710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1141/2268] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:23 2022] PUT /identity/v3/projects/e5f6acc6ed434c65964a5cdb81ae5e04/users/d77fe8d3046e4cfdb5b3f5bc15b31436/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:23.648622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.654477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.658624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.658918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.659756 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.679982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4318f31-ac84-457e-b220-9b9288f2cc16 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.694759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] MFA Rules not processed for user `d77fe8d3046e4cfdb5b3f5bc15b31436`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:23.698078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:23.775944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.776485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.776859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.777202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] PATH_INFO: `/v3/projects/12de18c1662c45e38619c52ba37e4744` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.778435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12de18c1662c45e38619c52ba37e4744)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.780429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.790340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1142/2269] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:23.795062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.806885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a26b74-ed86-4cf1-bee7-e3b5a8ddc58f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:23.811808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:23.817750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:23.817750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:23.817750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:23.817750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:23.817750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:23.900024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12de18c1662c45e38619c52ba37e4744', 'request_id': 'req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.905844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12de18c1662c45e38619c52ba37e4744', 'request_id': 'req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.912689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12de18c1662c45e38619c52ba37e4744', 'request_id': 'req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:23.941192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1143/2270] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:23 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:23.944867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:23.959795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:24.008366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4578c6-d804-48a7-8758-e9cff374fea8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.013254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.013881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.014185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.014453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.015074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.017090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.035178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] The token cache is being invalidate because project 12de18c1662c45e38619c52ba37e4744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:24.035468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.036101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.039351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1128/2271] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:23 2022] DELETE /identity/v3/projects/12de18c1662c45e38619c52ba37e4744 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:24.042071 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.043402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:24.105210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbd5bc-7d58-46e2-aba5-05d7bd66c438 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.109837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.110467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.110746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.111019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] PATH_INFO: `/v3/users/74809413f5da4de298f66d54d8a07a02` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.112154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74809413f5da4de298f66d54d8a07a02)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.114132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.245808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1144/2272] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:23 2022] POST /identity/v3/users => generated 298 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:24.246616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.248261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:24.260238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066f42-edc4-42ec-b17e-d35789266eca admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.264846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.265476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.265757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.266031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] PATH_INFO: `/v3/projects/4fbdbb1929714d309a40130cb7414d43` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.267164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fbdbb1929714d309a40130cb7414d43)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.269177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.454090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:24.468458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74809413f5da4de298f66d54d8a07a02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.471407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fbdbb1929714d309a40130cb7414d43', 'request_id': 'req-b2e8ef38-bc01-48d3-94a1-acd421439da2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.476140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74809413f5da4de298f66d54d8a07a02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.476910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74809413f5da4de298f66d54d8a07a02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.477643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74809413f5da4de298f66d54d8a07a02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.572546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fbdbb1929714d309a40130cb7414d43', 'request_id': 'req-b2e8ef38-bc01-48d3-94a1-acd421439da2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.579905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fbdbb1929714d309a40130cb7414d43', 'request_id': 'req-b2e8ef38-bc01-48d3-94a1-acd421439da2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.590798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] The token cache is being invalidate because project 4fbdbb1929714d309a40130cb7414d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:24.591084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.591724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.593243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1145/2273] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:24 2022] DELETE /identity/v3/projects/4fbdbb1929714d309a40130cb7414d43 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:24.594009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.595286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:24.666030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ef38-bc01-48d3-94a1-acd421439da2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.670943 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.671720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.671942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.672090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.672771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.674395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528d52b-ca2a-433f-9337-03d81612b531 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.685707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1146/2274] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:24.687006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.687288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.687561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.707495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:24.708745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:24.716726 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1147/2275] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:24.717473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.727364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24406075-7b4a-4d71-b1b4-f09360896d23 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.727537 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-07f96819-71be-4f61-99c3-056ee0bfad70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74809413f5da4de298f66d54d8a07a02'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:24.732079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.732758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.733038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.733329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] PATH_INFO: `/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6/users/d6820e6fa9ab431b94141736f4afce01/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.748022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1129/2276] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:24 2022] DELETE /identity/v3/users/74809413f5da4de298f66d54d8a07a02 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:24.749494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.749873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.750218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.751570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d11a25ecf6e49eca6650c87e48b98f6, user_id=d6820e6fa9ab431b94141736f4afce01, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.754265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.771156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:24.772382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:24.838038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1130/2277] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:24.843142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.853090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f715804-99fc-4edf-bc1d-1c68d57df478 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.859790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.860402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.860709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.860984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.861638 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:24.861942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.863762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.875764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1131/2278] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:24.880132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.890172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cfd57-4a08-4e3e-8001-fb4cc15c3099 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.895473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.896177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.896476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.896868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.897511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.900257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.905588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1148/2279] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:24 2022] PUT /identity/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6/users/d6820e6fa9ab431b94141736f4afce01/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:24.907179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:24.972399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ed6a8-d3e4-473c-98d7-d8efa24a78cd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:24.978598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:24.979221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:24.979499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:24.979770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:24.980424 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:24.982068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:24.992671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1149/2280] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:24.996760 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.006661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92789daf-492c-4f2c-900f-8996f489da21 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.011733 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.012408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.012716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.013118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] PATH_INFO: `/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6/users/d6820e6fa9ab431b94141736f4afce01/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.014986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d11a25ecf6e49eca6650c87e48b98f6, user_id=d6820e6fa9ab431b94141736f4afce01, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.015619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1132/2281] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:24 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:25.019280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.019621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4d3e2-f5aa-41c9-a5c8-e8b8c1670e2f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.043080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1150/2282] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:24 2022] PUT /identity/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6/users/d6820e6fa9ab431b94141736f4afce01/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:25.053705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.054113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.054606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.087662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] MFA Rules not processed for user `d6820e6fa9ab431b94141736f4afce01`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:25.088942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.099419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3a9d-6163-43cb-867f-fbd77b8712cb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.181011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.181648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.181932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.182102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1151/2283] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:25.182244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.183001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.185051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.194110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.198015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.256059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e663ce-9ae2-45d5-a2de-7daeae190ddc None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.260223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.260910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.261187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.261456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.263229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.331428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1133/2284] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:25 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:57:25.335081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.335400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.337585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.338019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.347594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1152/2285] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:25.348680 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.353707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868912e5-b193-484e-a821-57c4649358b6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.358076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.358721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.359138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.359396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] PATH_INFO: `/v3/projects/e95f4972414743c7ad09c28667b61260` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.360485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95f4972414743c7ad09c28667b61260)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.362061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90d6e5-c0fb-4b1c-b3d7-d664d525bb5a service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.362449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.367485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.368118 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.368381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.368704 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.369433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.371043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.397635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1153/2286] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:25.404802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.412468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a5f50-a30b-4592-9eb7-9e529966c6af admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.419734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.420394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.420803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.421066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.422136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.424861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.441596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1154/2287] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:25.442587 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.455416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0d0cd-b816-411b-9dcd-3920b33a8b09 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.462461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:25.463153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.463426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.463804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] PATH_INFO: `/v3/projects/dfb7eeaedda34cc4bc55302b56d64fe5/users/fea387580c7f4db5bc5e64db58b64e0f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.484346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb7eeaedda34cc4bc55302b56d64fe5, user_id=fea387580c7f4db5bc5e64db58b64e0f, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:25.488121 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a477c-d27e-412b-93e2-df86d381b474 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:25.659359 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1155/2288] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:25 2022] PUT /identity/v3/projects/dfb7eeaedda34cc4bc55302b56d64fe5/users/fea387580c7f4db5bc5e64db58b64e0f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:25.668369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95f4972414743c7ad09c28667b61260', 'request_id': 'req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:25.670504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0323d-e4ff-42bf-8313-7dc1b8dbabfc None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.670890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0323d-e4ff-42bf-8313-7dc1b8dbabfc None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.671265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0323d-e4ff-42bf-8313-7dc1b8dbabfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.675674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95f4972414743c7ad09c28667b61260', 'request_id': 'req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:25.682780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95f4972414743c7ad09c28667b61260', 'request_id': 'req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:25.683344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:25.712885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-49c0323d-e4ff-42bf-8313-7dc1b8dbabfc None None] MFA Rules not processed for user `fea387580c7f4db5bc5e64db58b64e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:25.714133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-49c0323d-e4ff-42bf-8313-7dc1b8dbabfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.784951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1156/2289] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:25.845687 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] The token cache is being invalidate because project e95f4972414743c7ad09c28667b61260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:25.846054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:25.847124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e42dad02-06c3-4dd9-8c92-a379dd60fdfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:25.849167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1134/2290] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:25 2022] DELETE /identity/v3/projects/e95f4972414743c7ad09c28667b61260 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:25.855514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.855800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.856113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.878516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:25.882958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.940269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1157/2291] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:57:25.948545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:25.948858 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:25.949199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:25.969305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:25.970507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:25.978933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1135/2292] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:25.983207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:25.993031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8487b05-080a-4306-80e5-b3270c7fcc39 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:25.999445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:26.008432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.008432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.008432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.008432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.008856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:26.008856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.009979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.025993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1158/2293] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:26.027270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.038240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc054ec-7a3f-470d-8b30-92adcea3de1b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.042313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.043032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.043268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.043545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.044173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.046092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.172491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1136/2294] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:26 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:26.176895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.187191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58998a06-2532-4673-bca9-61a894583cae admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.246309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.246916 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.247190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.247458 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.248129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.250143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.376231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1159/2295] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:26 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:26.378152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.389128 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67757db6-4773-4278-8577-18fcff536bb9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.392229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.394080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.394555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.395159 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.395515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.395715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.396362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.398543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.410865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1137/2296] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:26.415511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.425124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f0bd7-2b1c-4d3d-ada0-5fcd43baa3d7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.429702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.430286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.430560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.430831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.431568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.433177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.443342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1138/2297] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:57:26.447420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.452900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19d66d-2fee-4d75-b817-ba92e9dd3300 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.456781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.457410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c5613-bb45-4ec0-9c23-941ee65a2c80 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.457622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.457990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.458349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.460258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.462528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.463177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.463438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.463707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] PATH_INFO: `/v3/projects/a795f69f485a4f0994cf6b450891cd55/users/c1601276ca254f31b13b72af8e7c9576/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.479613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a795f69f485a4f0994cf6b450891cd55, user_id=c1601276ca254f31b13b72af8e7c9576, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.482595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.530602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.530602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4b72-6a0c-4959-a5af-529fd48f3a66 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.540165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1160/2298] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:57:26.600169 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1139/2299] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:26 2022] PUT /identity/v3/projects/a795f69f485a4f0994cf6b450891cd55/users/c1601276ca254f31b13b72af8e7c9576/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:57:26.612166 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.612633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.613588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.660234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] MFA Rules not processed for user `c1601276ca254f31b13b72af8e7c9576`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:26.661988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.747839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1161/2300] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:26.803194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:26.807478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:26.877023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36709826-26ad-49d2-9b57-19867b4478f1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:26.883828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:26.884946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:26.885469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:26.885976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:26.897293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:26.898801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:26.908828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1140/2301] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:27.064218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:28.062296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:28.073606 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3cc97-00cf-4d86-8ff6-2de660910569 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:28.077343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:28.077957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:28.078239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:28.078513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:28.080397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:28.141374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:28.142806 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b792827-ff38-401f-bf39-c8d9c9c791a9 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:28.152233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1162/2302] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:28.196604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:28.774906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:30.157663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:30.167083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d44c7-2211-4ffe-a448-a0c7322db33f service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:30.170776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:30.171371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:30.171648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:30.171920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:30.173676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:30.240645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:30.242180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:30.252641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1141/2303] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:31.404785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:31.469970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:31.989461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:32.790428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:32.902302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:33.470041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:43.019047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd93e6-745a-4633-bb04-fd60d3bb9de0 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:43.019613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd93e6-745a-4633-bb04-fd60d3bb9de0 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:43.019613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd93e6-745a-4633-bb04-fd60d3bb9de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:43.041004 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-67dd93e6-745a-4633-bb04-fd60d3bb9de0 None None] MFA Rules not processed for user `51729e82a3784d04aef89ed83f1576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:43.042240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-67dd93e6-745a-4633-bb04-fd60d3bb9de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:43.099327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1163/2304] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:43.843048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=20 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:46.967939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:46.969418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:47.056814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4848-17da-4cc9-a368-7d39a6c2fb96 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:47.061802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:47.062629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:47.062903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:47.063200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] PATH_INFO: `/v3/users/fc7c2f4fe9d54f7ab2debc508e4d4153` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:47.064381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7c2f4fe9d54f7ab2debc508e4d4153)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:47.066580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:47.277682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7c2f4fe9d54f7ab2debc508e4d4153'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:47.285287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7c2f4fe9d54f7ab2debc508e4d4153'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:47.285965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7c2f4fe9d54f7ab2debc508e4d4153'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:47.286573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7c2f4fe9d54f7ab2debc508e4d4153'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:47.380854 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7c2f4fe9d54f7ab2debc508e4d4153'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:47.400236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1142/2305] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:46 2022] DELETE /identity/v3/users/fc7c2f4fe9d54f7ab2debc508e4d4153 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:47.404637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:47.405168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:47.405265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:47.425858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:47.427093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:47.487271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1164/2306] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:47.499416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:47.501124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:47.559042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfae24-92ea-4c29-abfd-bb60ecd3cbe9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:47.562620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:47.563219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:47.563494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:47.563762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:47.573358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:47.574767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:47.584621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1143/2307] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:48.040484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:48.042302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:48.059585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11435a68-4bd8-44fb-85fc-efd7bda05eaf None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:48.061371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:48.062075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:48.062276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:48.062552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] PATH_INFO: `/v3/projects/33de9ced59b54846a2d5b81a1c7f7df0` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:48.063682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33de9ced59b54846a2d5b81a1c7f7df0)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:48.065856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:48.179134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33de9ced59b54846a2d5b81a1c7f7df0', 'request_id': 'req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.245753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33de9ced59b54846a2d5b81a1c7f7df0', 'request_id': 'req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.252475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33de9ced59b54846a2d5b81a1c7f7df0', 'request_id': 'req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.262854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] The token cache is being invalidate because project 33de9ced59b54846a2d5b81a1c7f7df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:48.263141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.263763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.265297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1165/2308] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:48 2022] DELETE /identity/v3/projects/33de9ced59b54846a2d5b81a1c7f7df0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:48.269232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:48.279518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74860ff-1d8a-41f7-ab9c-0eef7be9b3a1 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:48.298636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:48.336871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:48.337725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:48.338369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:48.338690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] PATH_INFO: `/v3/users/d77fe8d3046e4cfdb5b3f5bc15b31436` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:48.339834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77fe8d3046e4cfdb5b3f5bc15b31436)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:48.341804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:48.471166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77fe8d3046e4cfdb5b3f5bc15b31436'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.478110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77fe8d3046e4cfdb5b3f5bc15b31436'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.478786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77fe8d3046e4cfdb5b3f5bc15b31436'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.479406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77fe8d3046e4cfdb5b3f5bc15b31436'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.531512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77fe8d3046e4cfdb5b3f5bc15b31436'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:48.549778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1144/2309] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:48 2022] DELETE /identity/v3/users/d77fe8d3046e4cfdb5b3f5bc15b31436 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:48.990578 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:48.992540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:49.005907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940ccb-eb65-4324-bbc1-c5c6ba1361a4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:49.024938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:49.081658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:49.082330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:49.082802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:49.083264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] PATH_INFO: `/v3/projects/e5f6acc6ed434c65964a5cdb81ae5e04` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:49.084767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f6acc6ed434c65964a5cdb81ae5e04)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:49.086745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:49.177549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f6acc6ed434c65964a5cdb81ae5e04', 'request_id': 'req-47f773b7-f991-4f8d-9981-5adb2961057b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.263036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f6acc6ed434c65964a5cdb81ae5e04', 'request_id': 'req-47f773b7-f991-4f8d-9981-5adb2961057b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.269840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f6acc6ed434c65964a5cdb81ae5e04', 'request_id': 'req-47f773b7-f991-4f8d-9981-5adb2961057b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.281891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] The token cache is being invalidate because project e5f6acc6ed434c65964a5cdb81ae5e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:49.286352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.286352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.286352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1166/2310] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:48 2022] DELETE /identity/v3/projects/e5f6acc6ed434c65964a5cdb81ae5e04 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:49.288739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:49.290395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:49.303476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ac95a-87e7-40e3-be10-707d567f0bd5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:49.362274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:49.362909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:49.363191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:49.363569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] PATH_INFO: `/v3/users/d6820e6fa9ab431b94141736f4afce01` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:49.364868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6820e6fa9ab431b94141736f4afce01)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:49.366933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:49.535593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:49.538061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6820e6fa9ab431b94141736f4afce01'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.544822 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6820e6fa9ab431b94141736f4afce01'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.545488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6820e6fa9ab431b94141736f4afce01'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.546099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6820e6fa9ab431b94141736f4afce01'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.717959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c103b63c-0a2a-4b6c-a7aa-4a7009e9f000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6820e6fa9ab431b94141736f4afce01'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:49.748703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1145/2311] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:49 2022] DELETE /identity/v3/users/d6820e6fa9ab431b94141736f4afce01 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:50.674670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:50.676157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:50.689745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f773b7-f991-4f8d-9981-5adb2961057b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:50.758618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:50.759265 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:50.759549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:50.759823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] PATH_INFO: `/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:50.761130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d11a25ecf6e49eca6650c87e48b98f6)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:50.763081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:50.858016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:50.860044 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d11a25ecf6e49eca6650c87e48b98f6', 'request_id': 'req-b242af94-ec42-4d88-a335-bd1dc6593f81'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:50.920081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d11a25ecf6e49eca6650c87e48b98f6', 'request_id': 'req-b242af94-ec42-4d88-a335-bd1dc6593f81'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:50.927270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d11a25ecf6e49eca6650c87e48b98f6', 'request_id': 'req-b242af94-ec42-4d88-a335-bd1dc6593f81'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:50.940790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] The token cache is being invalidate because project 4d11a25ecf6e49eca6650c87e48b98f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:50.941724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:50.942583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b242af94-ec42-4d88-a335-bd1dc6593f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:50.944421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1167/2312] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:50 2022] DELETE /identity/v3/projects/4d11a25ecf6e49eca6650c87e48b98f6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:50.951881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:50.952797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:50.953320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:50.974375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:50.975599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:51.051440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1146/2313] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:51.061356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.061855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.062013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.094532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:51.095988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:51.108329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1168/2314] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:51.112510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.124171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3a85-1cbd-4123-84f4-6626bbd480c5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.129063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.129871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.130154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.130430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.133242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:51.133242 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.133613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.147665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1147/2315] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:51.156361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.166412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cccd-666b-46d3-8ef5-ad03a408e9d8 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.171207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.171819 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.172099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.172374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.179702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.179702 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.252393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1169/2316] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:51 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:51.253528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.264187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1e01c-422c-499a-b51a-15852d734b92 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.322360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.322884 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.323161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.323433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.324081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.326450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.451116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1148/2317] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:57:51 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:57:51.454753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.466533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673d20c-3a8c-47ce-ad94-691bc180a85d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.471710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.473014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.473299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.473572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.474268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.476949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.487808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1170/2318] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:51.495577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.506441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d43ee-35a0-4032-8f64-7da93c60d9a8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.511602 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.512219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.512498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.512802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.513456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.515061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4806c-a444-4204-89b6-b4b140f1b96a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.539298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1149/2319] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:57:51.543859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.555787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6c373-8a37-415a-abad-b86aed3b8295 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.561182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.561845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.562141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.562403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] PATH_INFO: `/v3/projects/7701e6492dd940df87340ba204be3138/users/5fe89d8592a44993838fc4acac83f974/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.585127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7701e6492dd940df87340ba204be3138, user_id=5fe89d8592a44993838fc4acac83f974, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.585127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.698040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1171/2320] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:57:51 2022] PUT /identity/v3/projects/7701e6492dd940df87340ba204be3138/users/5fe89d8592a44993838fc4acac83f974/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:51.708451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97b2d-67cc-4a6e-bb13-355eaa906eb2 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.708926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97b2d-67cc-4a6e-bb13-355eaa906eb2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.709292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c97b2d-67cc-4a6e-bb13-355eaa906eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.738206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f5c97b2d-67cc-4a6e-bb13-355eaa906eb2 None None] MFA Rules not processed for user `5fe89d8592a44993838fc4acac83f974`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:51.739469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f5c97b2d-67cc-4a6e-bb13-355eaa906eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:51.754063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:51.808277 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1150/2321] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:51.819527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:51.821352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:51.881595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf151f-84f4-42ae-9b7a-f1bd2987a923 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:51.885264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:51.885929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:51.886177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:51.886446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:51.895670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:51.897163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03ad3e-1217-4595-ac27-159743fd35c2 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:51.908473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1172/2322] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:54.831176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:54.831984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:54.833226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:54.859806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] MFA Rules not processed for user `5fe89d8592a44993838fc4acac83f974`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:54.862202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:54.880461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1151/2323] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:56.206137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:56.207731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:56.207731 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:56.229351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:56.231853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:56.296203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1173/2324] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:57:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:56.339607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:56.347195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:56.431773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db173c-b92b-44d4-b93f-9de7c35abb88 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:56.437576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:56.438312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:56.438778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:56.439259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:56.448581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:56.450232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66131f-8c41-416d-a0ea-0fc692f216dd service swift] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:56.460507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1152/2325] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:57:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:57:57.205584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:57.207167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:57.233401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:57.304712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeaf72-9e45-44ee-af69-f966fa82c7dd None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:57.420263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:57.420263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:57.420263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:57.420263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] PATH_INFO: `/v3/users/c1601276ca254f31b13b72af8e7c9576` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:57.421435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1601276ca254f31b13b72af8e7c9576)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:57.421435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:57.991161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1601276ca254f31b13b72af8e7c9576'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:57.992763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1601276ca254f31b13b72af8e7c9576'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:58.003322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1601276ca254f31b13b72af8e7c9576'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:58.113200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1601276ca254f31b13b72af8e7c9576'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:58.135231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1601276ca254f31b13b72af8e7c9576'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:58.146576 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1174/2326] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:57:57 2022] DELETE /identity/v3/users/c1601276ca254f31b13b72af8e7c9576 => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:58.158840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:58.159352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:58.159352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:58.182622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:58.184279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:58.278321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1153/2327] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:58.290106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:58.294378 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:58.377593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa63e-31a4-4681-84a1-61837034e662 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:58.385604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:58.386354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:58.386707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:58.387052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:58.414647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:58.417384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f5638-191a-476e-bfcf-2b41462ed402 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:58.430220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1175/2328] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:57:58.919356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:58.919356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:58.941924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bbc4a-12ef-48f3-8eb0-d21d6e604609 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:58.954694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:58.955309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:58.955591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:58.956047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] PATH_INFO: `/v3/projects/a795f69f485a4f0994cf6b450891cd55` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:58.957389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a795f69f485a4f0994cf6b450891cd55)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:58.959580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:59.077967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:59.096388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a795f69f485a4f0994cf6b450891cd55', 'request_id': 'req-a6787d09-09c2-478a-8499-d66d0198263d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:59.113709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a795f69f485a4f0994cf6b450891cd55', 'request_id': 'req-a6787d09-09c2-478a-8499-d66d0198263d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:59.123801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a795f69f485a4f0994cf6b450891cd55', 'request_id': 'req-a6787d09-09c2-478a-8499-d66d0198263d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:59.244336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] The token cache is being invalidate because project a795f69f485a4f0994cf6b450891cd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:57:59.246174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:59.246174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-a6787d09-09c2-478a-8499-d66d0198263d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:57:59.247426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1154/2329] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:57:58 2022] DELETE /identity/v3/projects/a795f69f485a4f0994cf6b450891cd55 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:57:59.252713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:59.253245 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:59.253734 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:59.293546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:59.294761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:59.350149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:57:59.415559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1176/2330] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:59.422932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:59.423784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:59.423784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:59.469510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:57:59.473248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:57:59.503122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1155/2331] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:57:59.515635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:59.522631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c95ef-a47b-4be9-94d6-e5ec6a2473e1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:59.528376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:59.529351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:59.529693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:59.529984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:59.530658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:57:59.530963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:59.532666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:57:59.555246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1177/2332] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:57:59.559576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:57:59.580530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cccfaf-6220-41c7-a9fb-5d0bec64d966 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:57:59.589417 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:57:59.591236 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:57:59.591746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:57:59.592257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:57:59.593404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:57:59.595566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:00.341027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1156/2333] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:57:59 2022] POST /identity/v3/projects => generated 369 bytes in 781 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:00.344549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:00.356282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29927af-55f5-47a0-9bce-8d7ad6901091 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:00.453873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:00.455059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:00.455059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:00.455059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:00.455871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:00.457870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:00.558416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1178/2334] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:00 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:00.562168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:00.573462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa862b5-e89b-44fd-8521-5f074f9f7894 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:00.579960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:00.581059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:00.581500 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:00.581928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:00.582740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:00.584525 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:00.614764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1157/2335] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:00.616522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:00.651508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f74b0-ed33-426c-9d82-f9793498dc09 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:00.658051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:00.658873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:00.659215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:00.659494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:00.661776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:00.663594 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919badb-9a9f-4920-af50-376ebceada21 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:00.683657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1179/2336] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:58:00.687927 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:00.699483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4355cb-e84c-4504-8a7d-d3bffb51d70a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:00.705032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:00.706235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:00.706781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:00.707529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] PATH_INFO: `/v3/projects/607263ff2dd04ed8b3daf8a4a062b48b/users/4ee306f25a4a41728edc00ce27297ec5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:00.729873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=607263ff2dd04ed8b3daf8a4a062b48b, user_id=4ee306f25a4a41728edc00ce27297ec5, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:00.733812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:00.827301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1158/2337] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:00 2022] PUT /identity/v3/projects/607263ff2dd04ed8b3daf8a4a062b48b/users/4ee306f25a4a41728edc00ce27297ec5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:58:00.844950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:00.844950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:00.844950 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:00.880831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] MFA Rules not processed for user `4ee306f25a4a41728edc00ce27297ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:00.882744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:00.960271 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1180/2338] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:00.973123 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:00.974963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:01.042461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadea6b-3599-405e-8de3-509f777c0163 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:01.046328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:01.046923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:01.047202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:01.047477 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:01.059859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:01.061357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2d490-8165-45ab-b79f-7042d0e8e69a service glance] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:01.071914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1159/2339] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:08.928530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:08.930051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:08.994303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:09.030305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35fa4-cad6-4b8b-821c-12fb23f04975 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:09.040835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:09.041780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:09.042226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:09.042656 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] PATH_INFO: `/v3/users/1693da72f2374aa29520e6b99a0ac26b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:09.043948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1693da72f2374aa29520e6b99a0ac26b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:09.047668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:09.289484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1693da72f2374aa29520e6b99a0ac26b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:09.290210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1693da72f2374aa29520e6b99a0ac26b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:09.297256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1693da72f2374aa29520e6b99a0ac26b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:09.467463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1693da72f2374aa29520e6b99a0ac26b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:09.483743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1693da72f2374aa29520e6b99a0ac26b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:09.491846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1181/2340] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:08 2022] DELETE /identity/v3/users/1693da72f2374aa29520e6b99a0ac26b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:09.498203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b730cf-79bb-4851-bf47-5534a3a55130 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:09.498645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b730cf-79bb-4851-bf47-5534a3a55130 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:09.500704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b730cf-79bb-4851-bf47-5534a3a55130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:09.521518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d3b730cf-79bb-4851-bf47-5534a3a55130 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:09.523213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d3b730cf-79bb-4851-bf47-5534a3a55130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:09.607732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1160/2341] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:09.607838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:09.607838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:09.661332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a964d30-3245-4b4d-90b8-0a6f158a0ed6 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:09.666415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:09.667270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:09.667687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:09.668057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:09.680307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:09.682020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:09.695485 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1182/2342] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:10.059558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:10.059939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:10.060420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:10.074616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:10.086710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:10.088210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:10.152711 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1161/2343] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:58:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:10.171029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:10.172630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:10.259534 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:10.281763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad804d2d-51ff-4a5b-8bf8-ce5823abbd10 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:10.292400 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:10.293056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:10.293434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:10.293645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:10.307735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:10.311048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:10.322139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1183/2344] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:58:10.324975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:10.328596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:10.358974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe62545-2ccf-4931-9421-da5e29faac94 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:10.364864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:10.365947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:10.366353 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:10.367368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] PATH_INFO: `/v3/projects/fdacd677c08040d58e81c8000607d909` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:10.370097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdacd677c08040d58e81c8000607d909)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:10.371046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:10.580399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdacd677c08040d58e81c8000607d909', 'request_id': 'req-bd73df18-a044-4897-8eba-45e7372980bd'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:10.591318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdacd677c08040d58e81c8000607d909', 'request_id': 'req-bd73df18-a044-4897-8eba-45e7372980bd'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:10.600864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdacd677c08040d58e81c8000607d909', 'request_id': 'req-bd73df18-a044-4897-8eba-45e7372980bd'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:10.710433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] The token cache is being invalidate because project fdacd677c08040d58e81c8000607d909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:10.712248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:10.720960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:10.729576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1162/2345] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:10 2022] DELETE /identity/v3/projects/fdacd677c08040d58e81c8000607d909 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:58:10.733817 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:10.751880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293558d-a440-4abc-9cd7-5e6ef67dea40 service swift] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:10.874413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:10.875073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:10.875351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:10.875623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] PATH_INFO: `/v3/users/09ead671782144fb8a1d4ebf2e2e30fc` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:10.877332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ead671782144fb8a1d4ebf2e2e30fc)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:10.884295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:11.138848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ead671782144fb8a1d4ebf2e2e30fc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:11.139723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ead671782144fb8a1d4ebf2e2e30fc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:11.155688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ead671782144fb8a1d4ebf2e2e30fc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:11.246110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ead671782144fb8a1d4ebf2e2e30fc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:11.265584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-430f55fe-62dd-482a-9d7d-30f6185761dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ead671782144fb8a1d4ebf2e2e30fc'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:11.276525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1184/2346] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:10 2022] DELETE /identity/v3/users/09ead671782144fb8a1d4ebf2e2e30fc => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:11.852007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:11.863437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:11.877776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73df18-a044-4897-8eba-45e7372980bd admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.011256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.012415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.012937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.013449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] PATH_INFO: `/v3/projects/e913c73ec03b406dbb34fc1bd5546034` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.016014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e913c73ec03b406dbb34fc1bd5546034)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.018287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.131530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:12.147700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e913c73ec03b406dbb34fc1bd5546034', 'request_id': 'req-3d31bb08-9e77-496f-a7e7-7c82081b0065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:12.157439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e913c73ec03b406dbb34fc1bd5546034', 'request_id': 'req-3d31bb08-9e77-496f-a7e7-7c82081b0065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:12.174799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e913c73ec03b406dbb34fc1bd5546034', 'request_id': 'req-3d31bb08-9e77-496f-a7e7-7c82081b0065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:12.247185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] The token cache is being invalidate because project e913c73ec03b406dbb34fc1bd5546034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:12.247589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:12.248206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3d31bb08-9e77-496f-a7e7-7c82081b0065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:12.250333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1163/2347] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:11 2022] DELETE /identity/v3/projects/e913c73ec03b406dbb34fc1bd5546034 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:12.255764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.258618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.258932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.287154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:12.287154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:12.366979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1185/2348] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:12.374779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.375296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.375836 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.403271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:12.404734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:12.414337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1164/2349] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:12.418765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.423713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:12.430544 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed115258-a614-47ad-aaf1-7776ceaff646 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.435664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.440857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.452855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1186/2350] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 07:58:12.457847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.476791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2da43-278a-4fd1-8b26-8fcbe0edc0ab None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.481680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.482301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.482590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.482880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.483508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.485691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.605300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1165/2351] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:12 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:12.610370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.623074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f9074-c7ce-424d-b998-35de4cc312c1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.690822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.691488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.691767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.692049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.692710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.694774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.790077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1187/2352] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:12 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:12.794399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.812205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723cae6-0818-4d30-8a32-36a3602e7824 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.818276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.818887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.819544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.819816 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.821372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.823076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.839295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1166/2353] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:12.844140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.859269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da48a01-a684-4f6d-83e9-f11bc311d55b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.878074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.880001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.880935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.881382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.882710 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.885644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31f1da-166f-4cc7-9e1e-74ccad1e468e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:12.908019 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1188/2354] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:12.910763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:12.925054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b84a0-81f4-45e2-aaae-bb0c3617ec86 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:12.930650 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:12.931716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:12.937135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:12.937135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] PATH_INFO: `/v3/projects/b6e2e9cd8f0f4d0a8844e47ff39aa12d/users/9b53aebd9309436abbf08fce43cdd19b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:12.952499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e2e9cd8f0f4d0a8844e47ff39aa12d, user_id=9b53aebd9309436abbf08fce43cdd19b, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:12.955748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.046875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1167/2355] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:12 2022] PUT /identity/v3/projects/b6e2e9cd8f0f4d0a8844e47ff39aa12d/users/9b53aebd9309436abbf08fce43cdd19b/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:13.058550 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025277-178a-4166-92e1-3253943319ee None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.063068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025277-178a-4166-92e1-3253943319ee None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.063068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05025277-178a-4166-92e1-3253943319ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.106506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-05025277-178a-4166-92e1-3253943319ee None None] MFA Rules not processed for user `9b53aebd9309436abbf08fce43cdd19b`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:13.108240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-05025277-178a-4166-92e1-3253943319ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:13.188104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1189/2356] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:13.189319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.293301 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d8020-d16f-4ec1-a9bc-d112b96f4fcb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.308319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.308319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.308319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.308319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.308686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.309219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.413144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1168/2357] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:13 2022] POST /identity/v3/projects => generated 365 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:13.414293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05025277-178a-4166-92e1-3253943319ee None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.429186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05025277-178a-4166-92e1-3253943319ee None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.488596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:13.496192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.496862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.497141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.497412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.498100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.500093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.598254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1190/2358] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:13 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:13.602230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.613577 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39acca-0dfe-4bb6-be98-47a6606b62f1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.618733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.619358 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.619665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.619980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.620693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.622317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.634731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1169/2359] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:13.638919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.650333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca210a0-b39c-467c-820c-8ff282b1f953 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.655586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.656585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.656904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.657306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] PATH_INFO: `/v3/projects/2c838bfc18ea49929c7e289eea5a0803/users/7d60fea4466442b58945049af091274d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.680172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c838bfc18ea49929c7e289eea5a0803, user_id=7d60fea4466442b58945049af091274d, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.683818 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.761695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1191/2360] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:13 2022] PUT /identity/v3/projects/2c838bfc18ea49929c7e289eea5a0803/users/7d60fea4466442b58945049af091274d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:13.766589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.777119 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44637b9-0cb7-4048-a178-7208e0a99ccb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.812314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:13.842223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.843150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.843446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.843955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.844675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.846708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3bc79-9a3e-46d6-9c63-daf3db2aa843 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.857304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1170/2361] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:13.861214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:13.872685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40937e-be90-4f7d-bab5-3ba66b001167 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:13.878383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:13.879043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.879316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.879583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] PATH_INFO: `/v3/projects/2c838bfc18ea49929c7e289eea5a0803/users/7d60fea4466442b58945049af091274d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.882644 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c838bfc18ea49929c7e289eea5a0803, user_id=7d60fea4466442b58945049af091274d, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:13.885672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:13.900469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1192/2362] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:13 2022] PUT /identity/v3/projects/2c838bfc18ea49929c7e289eea5a0803/users/7d60fea4466442b58945049af091274d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:13.911813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:13.914797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:13.915547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:13.950402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] MFA Rules not processed for user `7d60fea4466442b58945049af091274d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:13.951789 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:14.038461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1171/2363] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:14.050829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:14.053135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:14.131631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d765960-a9bb-4f86-aa98-1f9db57135ac admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:14.135673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:14.136307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:14.136571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:14.137076 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:14.157051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:14.159050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c877f6-0de3-470d-9772-25a5120aca1f service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:14.170574 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1193/2364] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:15.240881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:15.359189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873a3bf-520f-4f4c-a5c5-ea25e944951c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:15.367947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:15.368548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:15.368855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:15.369130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] PATH_INFO: `/v3/users/9b53aebd9309436abbf08fce43cdd19b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:15.372354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b53aebd9309436abbf08fce43cdd19b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:15.374459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:15.628797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b53aebd9309436abbf08fce43cdd19b'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:15.646918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:15.651260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b53aebd9309436abbf08fce43cdd19b'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:15.652382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b53aebd9309436abbf08fce43cdd19b'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:15.653369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b53aebd9309436abbf08fce43cdd19b'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:15.741019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b53aebd9309436abbf08fce43cdd19b'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:15.780351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1172/2365] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:15 2022] DELETE /identity/v3/users/9b53aebd9309436abbf08fce43cdd19b => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:15.786431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191b55-7996-4f37-be6f-817454776540 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:15.787128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191b55-7996-4f37-be6f-817454776540 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:15.787128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191b55-7996-4f37-be6f-817454776540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:15.807990 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9c191b55-7996-4f37-be6f-817454776540 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:15.809247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9c191b55-7996-4f37-be6f-817454776540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:15.883905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1194/2366] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:15.899134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:15.900851 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:15.978361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8a4e-0a99-4030-b366-93817cd3ab21 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:15.985356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:15.986044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:15.986374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:15.986652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:16.000251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:16.002206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:16.027282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1173/2367] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:16.509828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191b55-7996-4f37-be6f-817454776540 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:16.510653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9c191b55-7996-4f37-be6f-817454776540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:16.530585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191b55-7996-4f37-be6f-817454776540 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:16.536104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:16.538840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:16.538840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:16.538840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] PATH_INFO: `/v3/projects/b6e2e9cd8f0f4d0a8844e47ff39aa12d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:16.539148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e2e9cd8f0f4d0a8844e47ff39aa12d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:16.540964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:16.717459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:16.719000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e2e9cd8f0f4d0a8844e47ff39aa12d', 'request_id': 'req-10ff028f-a886-4a79-960d-e66bc96a9d4d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:16.815713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e2e9cd8f0f4d0a8844e47ff39aa12d', 'request_id': 'req-10ff028f-a886-4a79-960d-e66bc96a9d4d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:16.826820 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e2e9cd8f0f4d0a8844e47ff39aa12d', 'request_id': 'req-10ff028f-a886-4a79-960d-e66bc96a9d4d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:16.846894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] The token cache is being invalidate because project b6e2e9cd8f0f4d0a8844e47ff39aa12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:16.847419 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:16.851532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:16.859398 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1195/2368] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:16 2022] DELETE /identity/v3/projects/b6e2e9cd8f0f4d0a8844e47ff39aa12d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:16.863321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:16.880240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38fbd0-77a2-4022-873c-f7db896ee363 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:16.991625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:16.992793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:16.993268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:16.993972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] PATH_INFO: `/v3/users/7d60fea4466442b58945049af091274d` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:16.995657 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d60fea4466442b58945049af091274d)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:16.997920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:17.251461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d60fea4466442b58945049af091274d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:17.288438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d60fea4466442b58945049af091274d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:17.292047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d60fea4466442b58945049af091274d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:17.302582 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d60fea4466442b58945049af091274d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:17.404752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-edc9280d-1961-47fd-ad19-d552c1327191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d60fea4466442b58945049af091274d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:17.418199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:17.436697 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1174/2369] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:16 2022] DELETE /identity/v3/users/7d60fea4466442b58945049af091274d => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:18.017082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:18.018801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:18.042138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff028f-a886-4a79-960d-e66bc96a9d4d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:18.109741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:18.110597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.110976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.111373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] PATH_INFO: `/v3/projects/2c838bfc18ea49929c7e289eea5a0803` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.112600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c838bfc18ea49929c7e289eea5a0803)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:18.114719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:18.221868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c838bfc18ea49929c7e289eea5a0803', 'request_id': 'req-4d88e7b6-3965-4c6c-a045-283514234a13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:18.297641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c838bfc18ea49929c7e289eea5a0803', 'request_id': 'req-4d88e7b6-3965-4c6c-a045-283514234a13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:18.305582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c838bfc18ea49929c7e289eea5a0803', 'request_id': 'req-4d88e7b6-3965-4c6c-a045-283514234a13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:18.318143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] The token cache is being invalidate because project 2c838bfc18ea49929c7e289eea5a0803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:18.318431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:18.319144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4d88e7b6-3965-4c6c-a045-283514234a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:18.321568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1196/2370] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:18 2022] DELETE /identity/v3/projects/2c838bfc18ea49929c7e289eea5a0803 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:18.326756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.327241 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.327337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.361646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:18.362864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:18.494814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1175/2371] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:18.503802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.504080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.504351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.524364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:18.525681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:18.534375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1197/2372] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:18.539910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:18.550890 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c84f4-e38e-41ca-ab9a-9b8430ad448b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:18.558197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:18.569341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.570530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.570809 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.571521 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:18.576226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:18.577943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:18.606924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1176/2373] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:18.611443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:18.621840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4199c-ae3a-4a79-8298-9b56a215c991 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:18.627020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:18.627604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.627889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.628161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.628843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:18.630808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:18.746405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1198/2374] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:18 2022] POST /identity/v3/projects => generated 361 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:18.748916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:18.763220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3e03-7c7e-43d1-a6bb-e117b84c1882 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:18.850092 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:18.851315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:18.851710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:18.852102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:18.853419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:18.855722 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:18.986764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:18.998035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1177/2375] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:18 2022] POST /identity/v3/users => generated 281 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:19.006053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961558df-895a-49c1-aeca-46c9b49882fe admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:19.015369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:19.016392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.017610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.018497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.019465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:19.022084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:19.067185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:19.068037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1199/2376] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:19.080257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67901d-0cd7-4cf8-a676-14e41569c7c4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:19.087328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:19.092196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.092196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.092196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.092196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:19.092196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16941b-7f1a-4b81-b03e-fb4259ed1f00 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:19.113340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1178/2377] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:19.118292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:19.137801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8e3c-dd49-4c86-8f0f-77554ea033a8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:19.144308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14105-334c-4498-9532-d345286229af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:19.145317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14105-334c-4498-9532-d345286229af admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.145571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14105-334c-4498-9532-d345286229af admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.146202 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14105-334c-4498-9532-d345286229af admin admin] PATH_INFO: `/v3/projects/7e3fd2499e0c44a697f659e86237157f/users/69fb9ec8f1644d309ae12be8461c0de5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.171632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14105-334c-4498-9532-d345286229af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3fd2499e0c44a697f659e86237157f, user_id=69fb9ec8f1644d309ae12be8461c0de5, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:19.174960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14105-334c-4498-9532-d345286229af admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:19.314373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1200/2378] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:19 2022] PUT /identity/v3/projects/7e3fd2499e0c44a697f659e86237157f/users/69fb9ec8f1644d309ae12be8461c0de5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:19.316492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.317011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.317296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.355699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] MFA Rules not processed for user `69fb9ec8f1644d309ae12be8461c0de5`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:19.360138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:19.422038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:19.464992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1179/2379] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:19.519137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14105-334c-4498-9532-d345286229af admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:19.524689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7db14105-334c-4498-9532-d345286229af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:19.545037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:19.651876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14105-334c-4498-9532-d345286229af admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:19.656631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:19.657454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.658096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.658437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.668228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:19.670282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1830a5-225d-443a-bb36-b5fc402713bc service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:19.682797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1201/2380] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:19.700311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:19.832681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b7176-a5de-455a-8655-84b6f314077d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:19.834547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:19.837405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:19.837996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:19.856681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] PATH_INFO: `/v3/users/69fb9ec8f1644d309ae12be8461c0de5` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:19.858253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fb9ec8f1644d309ae12be8461c0de5)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:19.896902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:20.167248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fb9ec8f1644d309ae12be8461c0de5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:20.175695 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fb9ec8f1644d309ae12be8461c0de5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:20.176207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fb9ec8f1644d309ae12be8461c0de5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:20.177024 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fb9ec8f1644d309ae12be8461c0de5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:20.269627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fb9ec8f1644d309ae12be8461c0de5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:20.299541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1180/2381] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:19 2022] DELETE /identity/v3/users/69fb9ec8f1644d309ae12be8461c0de5 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:20.305720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6044-5121-4757-9275-943635934ac9 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:20.306210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6044-5121-4757-9275-943635934ac9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:20.306428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec6044-5121-4757-9275-943635934ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:20.340918 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-95ec6044-5121-4757-9275-943635934ac9 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:20.342592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-95ec6044-5121-4757-9275-943635934ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:20.452237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1202/2382] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:20.467620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:20.469272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:20.593950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde37b1-51ac-4555-8402-ca5341f9b0c6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:20.599899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:20.601177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:20.604941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:20.605305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:20.651308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:20.652997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e932a-bb81-499c-b7f0-c80c37d8cf96 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:20.666706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1181/2383] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:21.160926 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec6044-5121-4757-9275-943635934ac9 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:21.163579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-95ec6044-5121-4757-9275-943635934ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:21.189488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec6044-5121-4757-9275-943635934ac9 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:21.212212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:21.213296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.213685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.213965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] PATH_INFO: `/v3/projects/7e3fd2499e0c44a697f659e86237157f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.217551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3fd2499e0c44a697f659e86237157f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:21.219871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:21.484643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3fd2499e0c44a697f659e86237157f', 'request_id': 'req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:21.593007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3fd2499e0c44a697f659e86237157f', 'request_id': 'req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:21.601615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3fd2499e0c44a697f659e86237157f', 'request_id': 'req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:21.619925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] The token cache is being invalidate because project 7e3fd2499e0c44a697f659e86237157f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:21.620277 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:21.633472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-40b5cb4e-4f8d-4747-8f42-f857e5b0c0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:21.640484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1203/2384] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:21 2022] DELETE /identity/v3/projects/7e3fd2499e0c44a697f659e86237157f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:21.645465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.645932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.646295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.677661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:21.679640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:21.688180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:21.786818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1182/2385] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:21.794889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.795220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.795462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.816729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:21.817981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:21.832069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1204/2386] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:21.837931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:21.849282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185bb3c-d3ff-442c-ab86-0c883b3eee8e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:21.854089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:21.854696 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.854973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.855274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.855945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:21.856247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:21.857882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:21.870233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1183/2387] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:21.874490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:21.886236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cdd6c-a8eb-49df-8fe2-aee121e2f195 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:21.893498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:21.893498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:21.893498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:21.893498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:21.893989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:21.895727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:22.000301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:22.008109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1205/2388] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:21 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:22.019356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:22.030097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef4ea5-b212-43dd-9525-284b3c26216e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:22.099628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:22.100947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:22.101318 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:22.101483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:22.102144 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:22.104121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:22.214600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1184/2389] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:22 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:22.216141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:22.230981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc165f3-72d3-47ce-a7e4-da42dae2ff65 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:22.242857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:22.243508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:22.243779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:22.244195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:22.245007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:22.246650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:22.259982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1206/2390] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:22.263430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:22.275644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f332776-d9d6-4b7a-9ff4-5811b436fca2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:22.289349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:22.289349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:22.289349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:22.289349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:22.289774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:22.289774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b5b81-a677-4e9f-8d70-7455a1d4a53c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:22.300294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1185/2391] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:22.301703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:22.314580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613c77f-335f-4bca-af0b-9649c6ffe9a4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:22.321238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:22.321709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:22.322066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:22.322413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] PATH_INFO: `/v3/projects/7f7e8e3a96f24e73a9adccefb43c8cfd/users/15951af04e6040aaa2495f7eaa3c2a84/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:22.343385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7e8e3a96f24e73a9adccefb43c8cfd, user_id=15951af04e6040aaa2495f7eaa3c2a84, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:22.346722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcebf5e-db4b-4bdb-870e-e840de3ea6e7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:22.413708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1207/2392] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:22 2022] PUT /identity/v3/projects/7f7e8e3a96f24e73a9adccefb43c8cfd/users/15951af04e6040aaa2495f7eaa3c2a84/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:58:22.425269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc15ed-70fa-44be-b8a3-637b7ab7ff68 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:22.425636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc15ed-70fa-44be-b8a3-637b7ab7ff68 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:22.425984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc15ed-70fa-44be-b8a3-637b7ab7ff68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:22.463609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4bcc15ed-70fa-44be-b8a3-637b7ab7ff68 None None] MFA Rules not processed for user `15951af04e6040aaa2495f7eaa3c2a84`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:22.465245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4bcc15ed-70fa-44be-b8a3-637b7ab7ff68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:22.550702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1186/2393] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:23.303084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:23.306080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:23.307303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:23.313858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1208/2394] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:58:23 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:58:23.320941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:23.321479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:23.321479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:23.338781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:23.354194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:23.358781 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:23.490335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1187/2395] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:58:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:32.408703 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=15,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:33.247869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:33.249612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:33.314773 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c5890-556d-4c0f-aed0-37b9e0922c6d None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:33.320454 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:33.321349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.321629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.322313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.332983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:33.334593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094c531-66f5-47ee-8e5f-8f6772c4a858 service glance] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:33.345368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1209/2396] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:33.373786 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:33.391592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa39e44-374d-444b-8ac4-8659fbc63336 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:33.399484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:33.400173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.400717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.401344 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.431497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:33.433768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83658b9d-4580-4777-9ecb-b7156c29c448 service glance] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:33.449010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1188/2397] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:33.736280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.736780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.736890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.757111 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:33.758338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:33.799823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183ca95-b79a-436d-a9db-d661845b69a2 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.800148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183ca95-b79a-436d-a9db-d661845b69a2 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.800484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183ca95-b79a-436d-a9db-d661845b69a2 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.801926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1189/2398] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:58:33.807211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.807532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.808801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.817231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1210/2399] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:58:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:33.828636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:33.829931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:33.830643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:33.831964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:33.884262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1190/2400] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:58:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:33.896454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:33.897966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:33.903231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af3847-8645-4612-bffc-96c4fde1837f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:33.908559 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ccdc8-ab21-4429-8527-e3ad5247c2f4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:33.908897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:33.909503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.909790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.910060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.912302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:33.913041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:33.913322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:33.913592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:33.918335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:33.919930 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b93357-9731-47a9-ab90-a56458a66f4a service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:33.920508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:33.921959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef2423-f600-4667-b08a-b0acbd8810d1 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:33.928134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1211/2401] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:58:33.930930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1191/2402] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:34.013882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3026044-31b7-47a5-8246-32de397ca57b None None] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:34.014169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3026044-31b7-47a5-8246-32de397ca57b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:34.014445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3026044-31b7-47a5-8246-32de397ca57b None None] PATH_INFO: `/` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:34.015788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1212/2403] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 07:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 07:58:34.020794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea8c9e-a9fb-4946-ac63-f7fce1452d2d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:34.021018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea8c9e-a9fb-4946-ac63-f7fce1452d2d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:34.021292 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea8c9e-a9fb-4946-ac63-f7fce1452d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:34.041376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-dbea8c9e-a9fb-4946-ac63-f7fce1452d2d None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:34.042629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dbea8c9e-a9fb-4946-ac63-f7fce1452d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:34.051751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1192/2404] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 07:58:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:34.062776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3026044-31b7-47a5-8246-32de397ca57b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:34.074811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3026044-31b7-47a5-8246-32de397ca57b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:34.078470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:34.079060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:34.079333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:34.079599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:34.090236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:34.091646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:34.102343 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1213/2405] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:35.405911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:35.406502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:35.406502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:35.429912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:35.431439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:35.439607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1193/2406] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 07:58:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:35.456697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:35.469439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee27cf8-49de-41c6-96b9-a16bdfd2cee6 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:35.475632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:35.476596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:35.477056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:35.477500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:35.489517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:35.491815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113a95b-d646-428d-990d-a8fccc78fa6e service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:35.505492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1214/2407] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:58:36.381002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:36.382649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:36.418001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:36.418290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:36.418567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:36.439191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:36.440444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:36.448591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1215/2408] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:58:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:36.462149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:36.465166 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f8c0a-74f4-4bd8-b8a7-069440c8815c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:36.470058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:36.470200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:36.470724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:36.471034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:36.471307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:36.473207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:36.475099 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad92ce-52d1-4759-b817-aa3076b40620 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:36.480333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:36.482149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:36.482149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:36.482149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:36.485035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:36.486840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86a023-297a-4f28-bdd8-10e9d4944b0c service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:36.490511 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:36.491202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:36.492132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:36.498044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1194/2409] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:36.500434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1216/2410] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:58:37.179502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f963-8ab5-4296-9720-dd0f9f687be7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:37.179806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f963-8ab5-4296-9720-dd0f9f687be7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:37.180085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9f963-8ab5-4296-9720-dd0f9f687be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:37.204689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f9b9f963-8ab5-4296-9720-dd0f9f687be7 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:37.204689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f9b9f963-8ab5-4296-9720-dd0f9f687be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:37.210285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1195/2411] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 07:58:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:37.223791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:37.234830 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40402a47-0200-4253-86dd-c0ac9da15984 service swift] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:37.240110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:37.240796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:37.241603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:37.241872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:37.250124 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:37.251700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:37.261340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1217/2412] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:58:37.785865 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fd6d-76ab-4de4-bd0b-2202e3b97b2f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:37.786350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fd6d-76ab-4de4-bd0b-2202e3b97b2f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:37.786446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302fd6d-76ab-4de4-bd0b-2202e3b97b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:37.817623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0302fd6d-76ab-4de4-bd0b-2202e3b97b2f None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:37.819352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0302fd6d-76ab-4de4-bd0b-2202e3b97b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:37.835949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1196/2413] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 07:58:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:37.853069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:37.868243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de42a1a-0e49-428c-9d20-7d20dd026051 service swift] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:37.875681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:37.876551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:37.877320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:37.877727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:37.893281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:37.896071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:37.914481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1218/2414] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:58:39.330273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:39.330756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:39.335547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:39.361289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:39.362743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:39.371741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1197/2415] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:58:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:39.388406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:39.400156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887216c-ef4a-4239-acab-cc1867ecf475 service swift] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:39.405397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:39.406020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:39.406295 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:39.406568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:39.416021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:39.417633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e4c1e-4a71-464d-8940-787aa8e2db00 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:39.425670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1219/2416] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:58:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:58:40.232718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:40.245132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:40.317451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4a120-4549-4df2-aae0-6de354c726ae None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:40.322240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:40.323265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:40.323716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:40.324003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] PATH_INFO: `/v3/users/5fe89d8592a44993838fc4acac83f974` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:40.325204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe89d8592a44993838fc4acac83f974)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:40.327229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:40.533180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe89d8592a44993838fc4acac83f974'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:40.562309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe89d8592a44993838fc4acac83f974'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:40.568471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe89d8592a44993838fc4acac83f974'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:40.569798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe89d8592a44993838fc4acac83f974'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:40.683818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe89d8592a44993838fc4acac83f974'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:40.704989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1198/2417] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:40 2022] DELETE /identity/v3/users/5fe89d8592a44993838fc4acac83f974 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:40.710701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:40.711782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:40.715783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:40.739662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:40.741090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:40.848708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1220/2418] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:40.855742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:40.860802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:40.924248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3e337-ced4-408e-97ef-dbb239f1de70 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:40.928273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:40.928998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:40.929341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:40.929703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:40.939532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:40.941506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:40.965249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:40.982305 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1199/2419] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:41.294744 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:41.296686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:41.314376 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c5b6a-b6c5-45d7-8eed-d4e5797b6eac None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:41.321516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:41.322570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:41.323182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:41.323850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] PATH_INFO: `/v3/users/4ee306f25a4a41728edc00ce27297ec5` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:41.325530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee306f25a4a41728edc00ce27297ec5)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:41.328597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:41.425158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:41.428720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:41.456563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba299-5ecb-499d-8613-0c158e8e1806 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:41.472177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:41.473186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:41.473548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:41.473899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] PATH_INFO: `/v3/projects/7701e6492dd940df87340ba204be3138` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:41.490450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7701e6492dd940df87340ba204be3138)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:41.493349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:41.809457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7701e6492dd940df87340ba204be3138', 'request_id': 'req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:41.810535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee306f25a4a41728edc00ce27297ec5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:41.810535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee306f25a4a41728edc00ce27297ec5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:41.813236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee306f25a4a41728edc00ce27297ec5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:41.830317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7701e6492dd940df87340ba204be3138', 'request_id': 'req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:41.845714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7701e6492dd940df87340ba204be3138', 'request_id': 'req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.241317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee306f25a4a41728edc00ce27297ec5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.261649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee306f25a4a41728edc00ce27297ec5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.270053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1221/2420] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:41 2022] DELETE /identity/v3/users/4ee306f25a4a41728edc00ce27297ec5 => generated 0 bytes in 976 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:42.276380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:42.278174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.335407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:42.335717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] The token cache is being invalidate because project 7701e6492dd940df87340ba204be3138 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:42.335717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.335717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-fb8ac253-2c95-43fe-a3cf-4246f47bc9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.335717 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1200/2421] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:41 2022] DELETE /identity/v3/projects/7701e6492dd940df87340ba204be3138 => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:42.336937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3eac-5e04-4dd7-9d61-05bfe059c73d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.337225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3eac-5e04-4dd7-9d61-05bfe059c73d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.337504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea3eac-5e04-4dd7-9d61-05bfe059c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.360046 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8cea3eac-5e04-4dd7-9d61-05bfe059c73d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:42.364072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8cea3eac-5e04-4dd7-9d61-05bfe059c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.398556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d68e9-ca07-4925-8544-d43f5c5c8899 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:42.404047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:42.404854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.405386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.405885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] PATH_INFO: `/v3/users/ed5bde75f6d2477ab2747016ab5c309e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.408455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5bde75f6d2477ab2747016ab5c309e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:42.411513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:42.468712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1201/2422] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:42.470026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.470290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.470821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.492506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:42.493912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.501932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1202/2423] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:42.502719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:42.504013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.639168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229e105-da2d-4d3f-9752-16b72ece240f None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:42.654226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:42.654958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.655351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.655647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.676127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:42.676288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f3f17-b683-46fe-bf21-e0158185298f service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:42.680309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5bde75f6d2477ab2747016ab5c309e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.680951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5bde75f6d2477ab2747016ab5c309e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.688920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5bde75f6d2477ab2747016ab5c309e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.700915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1203/2424] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:42.702990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.703349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.703672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.754376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:42.766714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5bde75f6d2477ab2747016ab5c309e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.776642 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:42.790872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5bde75f6d2477ab2747016ab5c309e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:42.797101 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1204/2425] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:42.799902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1222/2426] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:42 2022] DELETE /identity/v3/users/ed5bde75f6d2477ab2747016ab5c309e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:42.801824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:42.812678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e92db-30dc-443f-a8a6-e4e1a36a709d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:42.831141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:42.972629 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:42.973716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:42.974215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:42.974648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:42.976551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:42.977195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:42.979345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:42.998622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1205/2427] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:43.000028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.021232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5e02e-65bc-4787-93e8-1c38547e7a4c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.044625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.044625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.044625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.044625 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.046799 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.052745 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:43.175366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.182386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1223/2428] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:42 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:43.194191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a3db2-b689-435d-b026-0b416a8d804c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.296304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.298063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.298366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.298772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.299450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.301898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:43.302624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.305946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.349809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca6151-dc7c-4c9b-a7a1-9a8d35d7e064 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.356897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.359747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.360901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.361518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] PATH_INFO: `/v3/projects/647a6443ec0c49e58d3cdb373d681c38` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.363629 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] RBAC: Authorizing `identity:delete_project(project_id=647a6443ec0c49e58d3cdb373d681c38)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.367989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:43.417217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1206/2429] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:43 2022] POST /identity/v3/users => generated 286 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:43.418264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.422797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.435684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81049e-f46b-49b8-ba4e-003d851f4bc0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.440885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.441531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.441808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.442081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] PATH_INFO: `/v3/projects/607263ff2dd04ed8b3daf8a4a062b48b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.443225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=607263ff2dd04ed8b3daf8a4a062b48b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.445394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:43.540441 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647a6443ec0c49e58d3cdb373d681c38', 'request_id': 'req-5690da82-7fbc-4b24-87a7-2176be91722f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.615270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '607263ff2dd04ed8b3daf8a4a062b48b', 'request_id': 'req-1b3f7e7a-014a-4605-a6f8-876c189ec949'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.623763 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '607263ff2dd04ed8b3daf8a4a062b48b', 'request_id': 'req-1b3f7e7a-014a-4605-a6f8-876c189ec949'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.639542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '607263ff2dd04ed8b3daf8a4a062b48b', 'request_id': 'req-1b3f7e7a-014a-4605-a6f8-876c189ec949'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.658872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647a6443ec0c49e58d3cdb373d681c38', 'request_id': 'req-5690da82-7fbc-4b24-87a7-2176be91722f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.667884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647a6443ec0c49e58d3cdb373d681c38', 'request_id': 'req-5690da82-7fbc-4b24-87a7-2176be91722f'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.680126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] The token cache is being invalidate because project 647a6443ec0c49e58d3cdb373d681c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:43.680413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.681126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.682941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1224/2430] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:43 2022] DELETE /identity/v3/projects/647a6443ec0c49e58d3cdb373d681c38 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:43.683722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.685179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.696334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:43.773310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] The token cache is being invalidate because project 607263ff2dd04ed8b3daf8a4a062b48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:43.773701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.774859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1b3f7e7a-014a-4605-a6f8-876c189ec949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:43.776644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1207/2431] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:43 2022] DELETE /identity/v3/projects/607263ff2dd04ed8b3daf8a4a062b48b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:43.781273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.781273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.781273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.822631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690da82-7fbc-4b24-87a7-2176be91722f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.823328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:43.824686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.869171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:43.933006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.933667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.934013 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.934350 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.935082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.936966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a1504-76ee-414e-badf-5a1b38645c23 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:43.952212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1225/2432] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:43.954872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25452-76c9-4a81-8d85-642a40a95d6d None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.955157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25452-76c9-4a81-8d85-642a40a95d6d None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.955432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a25452-76c9-4a81-8d85-642a40a95d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.967025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1208/2433] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:43.967025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:43.967025 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.977010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a1a25452-76c9-4a81-8d85-642a40a95d6d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:43.978319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a1a25452-76c9-4a81-8d85-642a40a95d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:43.983021 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34ff43-bfb4-44b9-acaa-954ebf02b72e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:43.986954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1226/2434] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:43.987942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:43.988261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.988535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.988680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:43.988894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.989007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:43.992799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:43.992799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:43.992799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43f38b-25b2-444c-a695-28883cbeba1d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.009489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1209/2435] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:44.011249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.011681 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.012028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.037636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:44.039020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:44.040257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:44.041909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:44.048952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1227/2436] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:44.049933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.054992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1210/2437] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:44.056006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.061393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6851bdd-33f6-457d-b8b4-7fc010e25367 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.068082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.069494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.069903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.070286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] PATH_INFO: `/v3/projects/576781ceab42470e9fd28a32ccb29e7f/users/0ae325c8314c47a384a83509e3c281a0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.070960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0f9d4-844c-4565-83f8-68b2ed2f41e6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.078994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.079906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.080770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.081244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.082102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:44.082402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.085818 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.104094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=576781ceab42470e9fd28a32ccb29e7f, user_id=0ae325c8314c47a384a83509e3c281a0, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.104873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1211/2438] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:44.105774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.108011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.116941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba15382-71ba-452a-a17b-0d30a11e9531 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.127426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.128126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.128403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.128847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.129682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:44.129844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.131959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.154921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1212/2439] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:44.158331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.168589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923128b6-a564-4f47-8dc1-3d7d14ef2be4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.173746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.174359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.174966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.175233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.176256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.178488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa7935-134a-425f-af96-564e738d5709 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.187324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1228/2440] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:44 2022] PUT /identity/v3/projects/576781ceab42470e9fd28a32ccb29e7f/users/0ae325c8314c47a384a83509e3c281a0/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:44.188683 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.292985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d76628-9654-40cf-8c29-e452646f7227 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.298456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.299101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.299379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.299966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.300387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.303063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.321283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1213/2441] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:44.321283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.321283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.321283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.331394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:44.362314 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:44.363625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:44.447241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1229/2442] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:44.447241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.507693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1214/2443] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:44.511378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.528911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc37ea0-9eba-486b-997f-3454c703bd44 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.567500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd1383-fb15-4e27-96d6-9c2d0b4e166d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.590462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.591665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.592114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.592555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.593787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.596798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.597401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:44.605299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.606064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.606487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.606765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.608098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.610151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.727832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1230/2444] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:44.727832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.729971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:44.754713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1215/2445] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:44 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:44.755738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.777813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb515588-91e2-413c-bdcd-e2348e727f11 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.784246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.784933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.785215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.785491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.786175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.789692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.802719 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1216/2446] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:44.804190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.825416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3984dd-9130-4767-a9c4-b39c524ab42f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.829984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6489-db02-4c2a-8818-027254e75186 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.831472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.832097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.832512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.832813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.833678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.835483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.838849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.839580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.839954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.840320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.852067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1217/2447] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:44.853734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.862665 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.864212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.865691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85535d4-3ef2-40de-b762-103c4c6c92f7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.874641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.875679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.876140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.876639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.877744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.880216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1231/2448] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:44.880336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.881936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.895976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1218/2449] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:44.896204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785662de-71ee-4fd2-aabd-3b41783ae7ea service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.897265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.922091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.937739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.937739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.937739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.937739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.937739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.938238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0bfe4-d31c-4531-b738-610802247026 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.940229 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.940938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.941223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.941502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] PATH_INFO: `/v3/projects/000fb9b5bcb542c8b7238adde32c3a12/users/ace808d428dd45cdb2eefc0735936933/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.945214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1232/2450] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:44.949855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:44.960377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=000fb9b5bcb542c8b7238adde32c3a12, user_id=ace808d428dd45cdb2eefc0735936933, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.960923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8c5d-432f-4fb7-8547-f4aeb6150602 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:44.964721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199816cb-0575-4954-bee8-c69bede20c94 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:44.965903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:44.972214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:44.972214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:44.972214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] PATH_INFO: `/v3/projects/17891e41ce9c4495a232a7665268391d/users/4315f71f642e42fea2352497369cbf74/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:44.985002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17891e41ce9c4495a232a7665268391d, user_id=4315f71f642e42fea2352497369cbf74, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:44.987640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd1aeb-3479-4ec0-be44-441f907f0e0a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:45.079148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1219/2451] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:44 2022] PUT /identity/v3/projects/000fb9b5bcb542c8b7238adde32c3a12/users/ace808d428dd45cdb2eefc0735936933/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:45.091271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:45.091644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:45.091824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:45.129906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] MFA Rules not processed for user `ace808d428dd45cdb2eefc0735936933`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:45.138360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:45.141292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1233/2452] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:44 2022] PUT /identity/v3/projects/17891e41ce9c4495a232a7665268391d/users/4315f71f642e42fea2352497369cbf74/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:45.156349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:45.156690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:45.156967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:45.202490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] MFA Rules not processed for user `4315f71f642e42fea2352497369cbf74`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:45.203783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:45.248845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1220/2453] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:45.266827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:45.269007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:45.287038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:45.336499 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1234/2454] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:45.371369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f861-01f9-480b-a6e7-37e12cb0c365 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:45.384677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:45.384677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:45.384677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:45.384677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:45.393343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:45.394866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:45.407037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1221/2455] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:45.498010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:45.500440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:45.517333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a91fc-20dc-4643-b2f4-d034a11d2303 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:45.523960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:45.525586 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:45.526066 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:45.526850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:45.554305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:45.556932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ba85f-1158-4ba8-b6a9-a7457baa60cb service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:45.584669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1235/2456] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:49.353462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:49.355043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:49.383701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:49.449857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c218474-8238-4d9a-a127-9b75920aaf37 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:49.450912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:49.451585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:49.451867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:49.452218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] PATH_INFO: `/v3/users/fea387580c7f4db5bc5e64db58b64e0f` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:49.453546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea387580c7f4db5bc5e64db58b64e0f)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:49.455610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:49.808201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea387580c7f4db5bc5e64db58b64e0f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.821251 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea387580c7f4db5bc5e64db58b64e0f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.822805 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea387580c7f4db5bc5e64db58b64e0f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.823653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea387580c7f4db5bc5e64db58b64e0f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.910084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea387580c7f4db5bc5e64db58b64e0f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:49.950931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1222/2457] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:49 2022] DELETE /identity/v3/users/fea387580c7f4db5bc5e64db58b64e0f => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:49.954601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:49.955821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:49.956167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:49.990460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:49.991709 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.083354 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1236/2458] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:50.094028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.095512 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.136889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.142321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.153790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72358c8-32ce-46b6-a932-1ff18186824a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.159588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.160908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.161327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.161697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] PATH_INFO: `/v3/users/ace808d428dd45cdb2eefc0735936933` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.163322 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace808d428dd45cdb2eefc0735936933)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.165436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.210360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b50c35-f0df-4cd5-bfc9-88d321fb9380 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.218587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.219237 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.219518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.219793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.231843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.235903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc151c-43e0-4626-b5c9-39ecd2bdf2b6 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.255916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1223/2459] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:50.424392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace808d428dd45cdb2eefc0735936933'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.425599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace808d428dd45cdb2eefc0735936933'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.435356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace808d428dd45cdb2eefc0735936933'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.515828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace808d428dd45cdb2eefc0735936933'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.536612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace808d428dd45cdb2eefc0735936933'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:50.545898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1237/2460] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:58:50 2022] DELETE /identity/v3/users/ace808d428dd45cdb2eefc0735936933 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:50.550364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.550880 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.550979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.575117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:50.575117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.639413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1224/2461] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:50.650923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.652455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.692680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:50.716014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:50.728517 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13c992-e5fc-4f26-aefa-03305d33289f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.732120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.732759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.733037 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.733358 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.743156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.744679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:50.764859 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1238/2462] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:50.893343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:50.894982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:50.912743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58796-585b-4de9-9690-a7c85e90b893 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:50.918998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:50.919859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:50.920244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:50.920669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] PATH_INFO: `/v3/projects/dfb7eeaedda34cc4bc55302b56d64fe5` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:50.922050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfb7eeaedda34cc4bc55302b56d64fe5)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:50.924530 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:51.051298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfb7eeaedda34cc4bc55302b56d64fe5', 'request_id': 'req-1458c67a-b65c-4412-a0e2-6a8ed50811ae'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:51.058146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfb7eeaedda34cc4bc55302b56d64fe5', 'request_id': 'req-1458c67a-b65c-4412-a0e2-6a8ed50811ae'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:51.073047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfb7eeaedda34cc4bc55302b56d64fe5', 'request_id': 'req-1458c67a-b65c-4412-a0e2-6a8ed50811ae'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:51.176011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] The token cache is being invalidate because project dfb7eeaedda34cc4bc55302b56d64fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:51.176011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:51.176011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1458c67a-b65c-4412-a0e2-6a8ed50811ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:51.183864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1225/2463] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:50 2022] DELETE /identity/v3/projects/dfb7eeaedda34cc4bc55302b56d64fe5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:51.729298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:51.730788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:51.752716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98a47-ba0c-4b23-b847-8a637f96fdde service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:51.819254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:51.820478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:51.820933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:51.821316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] PATH_INFO: `/v3/projects/000fb9b5bcb542c8b7238adde32c3a12` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:51.823699 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=000fb9b5bcb542c8b7238adde32c3a12)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:51.826313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:51.959289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000fb9b5bcb542c8b7238adde32c3a12', 'request_id': 'req-f6c62240-444c-4b3a-a357-d5402f615105'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:52.083689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000fb9b5bcb542c8b7238adde32c3a12', 'request_id': 'req-f6c62240-444c-4b3a-a357-d5402f615105'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:52.090728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000fb9b5bcb542c8b7238adde32c3a12', 'request_id': 'req-f6c62240-444c-4b3a-a357-d5402f615105'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:52.103648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] The token cache is being invalidate because project 000fb9b5bcb542c8b7238adde32c3a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:58:52.103648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:52.103648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-f6c62240-444c-4b3a-a357-d5402f615105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:58:52.105471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1239/2464] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:58:51 2022] DELETE /identity/v3/projects/000fb9b5bcb542c8b7238adde32c3a12 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:52.110268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.110768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.110850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.137630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:52.140307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:52.229296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1226/2465] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:52.241062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.241430 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.241639 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.262645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:52.263876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:52.274258 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1240/2466] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:52.279135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.294068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c99e2-d968-45d3-be48-7f97edd0112e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.301051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.301766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.302676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.302983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.309145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:58:52.309145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.309145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.329061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.329271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1227/2467] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:58:52.347191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670c67d-bdf7-4627-9e62-3fb78013f679 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.353171 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.353968 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.354404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.354682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.355535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.357739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af0e76-6430-4717-b0d6-c868c1092585 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.487341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1241/2468] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:52.492061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.509420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23ccd-b4b2-47b1-9aaa-bad51e5313cc admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.588609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.589415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.589744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.590396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.591195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.594096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.690555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1228/2469] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:52.693045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.700539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.700539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.703727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.721321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a9c-dffa-43d6-87b8-3a682215fd00 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.734546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.735178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.735461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.735782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.736458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.738375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] MFA Rules not processed for user `0ae325c8314c47a384a83509e3c281a0`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:52.738857 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.739815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:52.751830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1229/2470] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:58:52.755892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.767207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffe2f2-f485-4cd4-9a0c-76f6b188e423 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.773536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.774160 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.774433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.774751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.775434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.777084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.792623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1230/2471] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:58:52.793788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.805243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338cc2d-155a-41e9-bfe6-26720736d547 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.809938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.810612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.810915 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.811545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] PATH_INFO: `/v3/projects/c534e991899e457c8b41e1dbc12a6670/users/5a8de6b6200840dbb9c4d7a3b728f9c2/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.823021 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1242/2472] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:52.824553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:52.826028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:52.829741 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c534e991899e457c8b41e1dbc12a6670, user_id=5a8de6b6200840dbb9c4d7a3b728f9c2, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.832466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c53a6-3701-43f2-aa79-160ec1406648 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.841178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739761a-a4c6-4d0e-b785-11d304bf3112 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:52.846322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:52.846768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:52.847304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.847582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.847853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.848486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:52.850523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:52.927319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1231/2473] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:52 2022] PUT /identity/v3/projects/c534e991899e457c8b41e1dbc12a6670/users/5a8de6b6200840dbb9c4d7a3b728f9c2/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:52.933498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a98d-d357-4587-874e-257075a997bb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:52.933913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a98d-d357-4587-874e-257075a997bb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:52.934263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a98d-d357-4587-874e-257075a997bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:52.952723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:52.957595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-deb1a98d-d357-4587-874e-257075a997bb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:52.958798 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-deb1a98d-d357-4587-874e-257075a997bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:52.961193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1243/2474] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:58:52.965102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.018889 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1232/2475] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:58:53.028634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1a98d-d357-4587-874e-257075a997bb None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.029022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503cf16-ac45-4a20-94f1-f14b514d2678 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.030135 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-deb1a98d-d357-4587-874e-257075a997bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:53.033969 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.034579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.034856 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.035283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.035784 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.038235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.086748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1a98d-d357-4587-874e-257075a997bb None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.090487 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.091089 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.091362 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.091633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.101298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.102707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.113082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1233/2476] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:58:53.124931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1244/2477] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:58:52 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:58:53.130906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.155302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51495073-339d-428e-aa6b-7dae2a50b5b8 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.160934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.161548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.161826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.162098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.163278 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.165207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.177164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1234/2478] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:53.181808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.196582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214ffd6-36f6-4239-bc8a-e1b59d04491b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.203847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.204697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.205068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.205413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] PATH_INFO: `/v3/projects/daa7ded529644ce7866f85298f192831/users/de771603647a4445bac640f6cec4c3a4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.223855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa7ded529644ce7866f85298f192831, user_id=de771603647a4445bac640f6cec4c3a4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.227353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.334705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1245/2479] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:53 2022] PUT /identity/v3/projects/daa7ded529644ce7866f85298f192831/users/de771603647a4445bac640f6cec4c3a4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:53.338807 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.350482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9c0c6-b6a6-4a31-b6b4-4c650bd960a8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.493045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.493674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.493954 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.494227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.494985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.502035 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03642981-1365-4cb8-baf3-4cf536f7346f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.517448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1235/2480] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:58:53.525814 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:53.532193 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e258f-1b5c-4737-ad31-d7500d3f1a5e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:53.540219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:53.540929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:53.541214 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:53.541489 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] PATH_INFO: `/v3/projects/daa7ded529644ce7866f85298f192831/users/de771603647a4445bac640f6cec4c3a4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:53.543804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa7ded529644ce7866f85298f192831, user_id=de771603647a4445bac640f6cec4c3a4, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:53.546638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:53.572723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1246/2481] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:58:53 2022] PUT /identity/v3/projects/daa7ded529644ce7866f85298f192831/users/de771603647a4445bac640f6cec4c3a4/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:58:54.672574 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486d91c-1448-4fbf-be67-27c503764513 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:54.673136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486d91c-1448-4fbf-be67-27c503764513 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:54.673232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486d91c-1448-4fbf-be67-27c503764513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:54.715271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:58:54.728285 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-1486d91c-1448-4fbf-be67-27c503764513 None None] MFA Rules not processed for user `5a8de6b6200840dbb9c4d7a3b728f9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:54.730529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1486d91c-1448-4fbf-be67-27c503764513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:54.822316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1236/2482] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:58:54.830263 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:58:54.832900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:54.962211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b47047-db1e-418b-8ece-1b1b444e2623 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:58:54.967671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:58:54.968546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:54.968946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:54.969316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:54.981937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:58:54.983652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:58:54.998213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1247/2483] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:58:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:58:57.129801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4f42-e346-4d1a-8a7e-42db52c86638 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:58:57.129801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4f42-e346-4d1a-8a7e-42db52c86638 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:58:57.129801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4f42-e346-4d1a-8a7e-42db52c86638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:58:57.155868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-921a4f42-e346-4d1a-8a7e-42db52c86638 None None] MFA Rules not processed for user `5a8de6b6200840dbb9c4d7a3b728f9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:58:57.167625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-921a4f42-e346-4d1a-8a7e-42db52c86638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:58:57.195647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1237/2484] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:58:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:59:00.649878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:00.652108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:00.665862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:00.770381 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d451d-f82c-47f6-8ee5-ab92bb06248d service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:00.776687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:00.777374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:00.777658 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:00.777936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] PATH_INFO: `/v3/users/4315f71f642e42fea2352497369cbf74` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:00.779113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4315f71f642e42fea2352497369cbf74)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:00.781120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:00.991147 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:00.991632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:00.991729 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:01.006616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4315f71f642e42fea2352497369cbf74'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:01.007946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4315f71f642e42fea2352497369cbf74'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:01.017822 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4315f71f642e42fea2352497369cbf74'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:01.040198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] MFA Rules not processed for user `de771603647a4445bac640f6cec4c3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:01.041714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.129654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4315f71f642e42fea2352497369cbf74'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:01.136486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1238/2485] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:01.149082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c2d70c64-ed76-4cb8-bb97-da204dfac582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4315f71f642e42fea2352497369cbf74'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:01.156755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1248/2486] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:59:00 2022] DELETE /identity/v3/users/4315f71f642e42fea2352497369cbf74 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:01.159859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:01.161356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.161513 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:01.161951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:01.163000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:01.192342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:01.194151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.227179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:01.269239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770f4d8-e3ba-4b8a-9e11-af5ac23a4cc5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:01.269466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1249/2487] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:01.279519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:01.281225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.321185 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:01.350528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:01.351272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:01.351472 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:01.351637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:01.354303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.362172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ef4d5-ac3b-4bf7-9ab5-4b1ed754218f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:01.369038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:01.369975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:01.370257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:01.370563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:01.382793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:01.384123 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:01.397217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1250/2488] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:59:01.446834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:01.450937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f095f-6ff9-4767-b415-d52a03de1b69 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:01.465371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1239/2489] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:01.925723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:01.927508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:01.942029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77fc5-e5ea-438b-8011-7cea76002bc8 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:01.947992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:01.948872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:01.949242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:01.949627 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] PATH_INFO: `/v3/projects/17891e41ce9c4495a232a7665268391d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:01.959753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17891e41ce9c4495a232a7665268391d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:01.964616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.073523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17891e41ce9c4495a232a7665268391d', 'request_id': 'req-ccee6ed1-4072-4667-9166-5fd1904a4773'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:02.155366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17891e41ce9c4495a232a7665268391d', 'request_id': 'req-ccee6ed1-4072-4667-9166-5fd1904a4773'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:02.166553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17891e41ce9c4495a232a7665268391d', 'request_id': 'req-ccee6ed1-4072-4667-9166-5fd1904a4773'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:02.177428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] The token cache is being invalidate because project 17891e41ce9c4495a232a7665268391d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:59:02.177714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:02.178372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ccee6ed1-4072-4667-9166-5fd1904a4773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:02.179861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1251/2490] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:59:01 2022] DELETE /identity/v3/projects/17891e41ce9c4495a232a7665268391d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:02.184770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.185244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.185355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.205659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:02.206792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:02.275196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1240/2491] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:02.283199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.283478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.283769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.305157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:02.306389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:02.314525 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1252/2492] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:02.319550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.329600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a1a2-d6a6-44b8-bad2-c281b111150b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.334990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.335571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.335918 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.336334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.337047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:59:02.337351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.339141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.342005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:02.351839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1241/2493] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:59:02.359928 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.370450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7be85-7082-462b-83ce-651ea162ef36 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.380415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.381643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.382174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.382308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.382842 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.384821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.472064 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1253/2494] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:59:02 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:59:02.474920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.485323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10890b69-5431-4717-96e9-c20ec88f97eb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.563409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.564028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.564298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.564568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.565239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.567262 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.684892 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1242/2495] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:59:02 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:59:02.686999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.697203 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca32be6-883e-4661-8939-1a758dbb99ba admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.702584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.703269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.703614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.703953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.704788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.706463 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.720393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1254/2496] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:59:02.723684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.734999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c812f7-ac42-4008-affe-5da66af208f5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.742817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.742817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.742817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.742817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.749454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.749454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c9d2cd-012d-4d3f-8439-d7c8cfeea218 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.760944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1243/2497] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:59:02.762435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:02.773995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de55143-6ed2-4641-89c9-5038f4c7ba8b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:02.779160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:02.779946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.780300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.780686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] PATH_INFO: `/v3/projects/32dee2c6dfd447488ae7077af3cca683/users/e87d0ca4791844f79667af27048e7704/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.799371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32dee2c6dfd447488ae7077af3cca683, user_id=e87d0ca4791844f79667af27048e7704, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:02.802128 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:02.879612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1255/2498] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:59:02 2022] PUT /identity/v3/projects/32dee2c6dfd447488ae7077af3cca683/users/e87d0ca4791844f79667af27048e7704/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:02.891086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a38cc-bfe8-46e1-a6b9-543d65cf2261 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:02.891378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a38cc-bfe8-46e1-a6b9-543d65cf2261 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:02.891659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a38cc-bfe8-46e1-a6b9-543d65cf2261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:02.926074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-dc6a38cc-bfe8-46e1-a6b9-543d65cf2261 None None] MFA Rules not processed for user `e87d0ca4791844f79667af27048e7704`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:02.927343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dc6a38cc-bfe8-46e1-a6b9-543d65cf2261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:03.010112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1244/2499] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:03.022167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:03.024282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:03.088268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317503e-d42b-4e8f-924a-c02e25abd782 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:03.092851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:03.093681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:03.094192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:03.094741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:03.101415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:03.105522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:03.107078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569224f0-da9d-40f6-9526-981dee6c705b service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:03.117618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1256/2500] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:04.532631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:04.533449 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:04.533771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:04.557870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] MFA Rules not processed for user `0ae325c8314c47a384a83509e3c281a0`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:04.559502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:04.624628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1245/2501] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:04.914045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:04.914045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:04.914045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:04.935603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:04.936905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:05.033104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1257/2502] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 07:59:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:05.062946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:05.064636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:05.079736 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:05.199279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111c9c-465d-4b14-b2aa-204bbfd6a362 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:05.212377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:05.213126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:05.213531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:05.213872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:05.223930 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:05.225750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9609c1-dba0-4b4f-b0bc-d54fa0596d78 service swift] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:05.236903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1246/2503] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:59:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 07:59:15.020364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=28 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:16.429767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=28 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:28.930098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:28.931729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:29.003442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f7bfa-87b1-499f-9eef-46ef28435cd4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:29.008032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:29.008708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:29.008958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:29.009226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] PATH_INFO: `/v3/users/15951af04e6040aaa2495f7eaa3c2a84` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:29.010346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15951af04e6040aaa2495f7eaa3c2a84)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:29.013864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:29.189802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15951af04e6040aaa2495f7eaa3c2a84'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:29.190484 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15951af04e6040aaa2495f7eaa3c2a84'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:29.201959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15951af04e6040aaa2495f7eaa3c2a84'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:29.272227 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15951af04e6040aaa2495f7eaa3c2a84'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:29.294715 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15951af04e6040aaa2495f7eaa3c2a84'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:29.304543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1258/2504] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:59:28 2022] DELETE /identity/v3/users/15951af04e6040aaa2495f7eaa3c2a84 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:29.306300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:29.306777 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:29.307041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:29.327011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:29.328295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:29.406995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1247/2505] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:29.417119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:29.419180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:29.495786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea27f4-4cbe-45f5-bc76-8869fbd61f75 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:29.501186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:29.501794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:29.502070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:29.502339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:29.512387 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:29.515080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b586-2f13-423b-8e8b-d5a5e4404c8c service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:29.525119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1259/2506] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:30.046191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.047785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:30.063199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4c8f4-d447-49df-8eb9-68bfe95c678d None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.071933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.072998 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.073338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.076850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] PATH_INFO: `/v3/projects/7f7e8e3a96f24e73a9adccefb43c8cfd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.078509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7e8e3a96f24e73a9adccefb43c8cfd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.080960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.289675 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7e8e3a96f24e73a9adccefb43c8cfd', 'request_id': 'req-e0318d95-d082-48e3-a65e-7a9824fe964a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:30.302240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7e8e3a96f24e73a9adccefb43c8cfd', 'request_id': 'req-e0318d95-d082-48e3-a65e-7a9824fe964a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:30.311469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7e8e3a96f24e73a9adccefb43c8cfd', 'request_id': 'req-e0318d95-d082-48e3-a65e-7a9824fe964a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:30.415302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] The token cache is being invalidate because project 7f7e8e3a96f24e73a9adccefb43c8cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:59:30.415658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:30.416463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-e0318d95-d082-48e3-a65e-7a9824fe964a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:30.419468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1248/2507] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:59:30 2022] DELETE /identity/v3/projects/7f7e8e3a96f24e73a9adccefb43c8cfd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:30.425423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57df38-452a-46c0-b776-feb717fa062b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.425978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57df38-452a-46c0-b776-feb717fa062b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.425978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57df38-452a-46c0-b776-feb717fa062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.452297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-df57df38-452a-46c0-b776-feb717fa062b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:30.454069 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-df57df38-452a-46c0-b776-feb717fa062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:30.480178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:30.534946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1260/2508] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:30.544090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.544090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.544090 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.564288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:30.565527 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:30.573542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1249/2509] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:30.577757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57df38-452a-46c0-b776-feb717fa062b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.587536 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57df38-452a-46c0-b776-feb717fa062b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.592911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.593514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.593791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.594231 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.594877 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 07:59:30.595177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.596887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.609592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1261/2510] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 07:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 07:59:30.614132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.624796 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c207796-5db4-4a07-bb1a-3b2531f76e39 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.629617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.630382 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.630666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.630925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.631543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.633607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.725464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1250/2511] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:59:30 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:59:30.730172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.740909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee6021-51ac-4472-9890-ba5f8b16e7b7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.805330 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.805952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.806237 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.806515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.807145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.814735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.889262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1262/2512] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:59:30 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:59:30.890802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.901839 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd3227-7ab2-4bad-b3a7-b5122b0705ca admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.906771 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.907423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.907970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.908243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.908995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.910663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.924901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1251/2513] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:59:30.926435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.937663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366b13b-8a7f-4787-9244-6e678e276b20 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.942379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.942984 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.943259 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.943531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.944177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.945849 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e6aea-cf15-43be-9c36-21cb54049240 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:30.957366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1263/2514] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:59:30.961641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:30.980820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84876-b1be-4a95-8a57-9f1d37441e1b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:30.980820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:30.980820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:30.980820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:30.981888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] PATH_INFO: `/v3/projects/4b41efe770974b3aa9a34e21dc2ea5d2/users/b3fa420e4c0049b9ae5cee7717422cda/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:30.996399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b41efe770974b3aa9a34e21dc2ea5d2, user_id=b3fa420e4c0049b9ae5cee7717422cda, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:30.999162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:31.121050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1252/2515] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:59:30 2022] PUT /identity/v3/projects/4b41efe770974b3aa9a34e21dc2ea5d2/users/b3fa420e4c0049b9ae5cee7717422cda/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 07:59:31.126170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:31.126453 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:31.126723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:31.146209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:31.147531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:31.208071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1264/2516] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:31.218612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:31.220466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:31.242283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:31.277479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcf77-5ec8-4080-8e0d-cac306b84bd7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:31.282079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:31.282944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:31.283224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:31.283498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:31.292531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:31.294515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e979b06-ec95-4486-a572-984b70555faa service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:31.303943 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1253/2517] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:34.407001 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:34.411518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:34.428292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818095b-a580-45aa-8aa5-bfbeb5ab2e9e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:34.443110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:34.446106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:34.446556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:34.447089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] PATH_INFO: `/v3/users/e87d0ca4791844f79667af27048e7704` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:34.449220 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87d0ca4791844f79667af27048e7704)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:34.458471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:34.506633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:34.745905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87d0ca4791844f79667af27048e7704'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:34.747581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87d0ca4791844f79667af27048e7704'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:34.761568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87d0ca4791844f79667af27048e7704'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:34.855177 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87d0ca4791844f79667af27048e7704'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:34.872546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87d0ca4791844f79667af27048e7704'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:34.879922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1265/2518] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 07:59:34 2022] DELETE /identity/v3/users/e87d0ca4791844f79667af27048e7704 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:34.884941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:34.886950 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:34.887443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:34.924708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:34.927359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:35.026536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1254/2519] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:35.065929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:35.068200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:35.191015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc025-3e9b-4332-b78c-8aa0c02e4c4b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:35.196046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:35.196841 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:35.197055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:35.197333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:35.209615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:35.212091 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943c9dd-b271-4795-92c4-b40bf91599a7 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:35.235953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1266/2520] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:35.953586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:35.955311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:35.970244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3eb35-616c-4e73-9709-effb37d2b8c0 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:35.976560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:35.977550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:35.978043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:35.978522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] PATH_INFO: `/v3/projects/32dee2c6dfd447488ae7077af3cca683` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:35.979983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] RBAC: Authorizing `identity:delete_project(project_id=32dee2c6dfd447488ae7077af3cca683)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:35.982762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:36.128221 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32dee2c6dfd447488ae7077af3cca683', 'request_id': 'req-d3a6988c-6037-4021-bec6-abda6f27d25f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:36.136544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32dee2c6dfd447488ae7077af3cca683', 'request_id': 'req-d3a6988c-6037-4021-bec6-abda6f27d25f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:36.146099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32dee2c6dfd447488ae7077af3cca683', 'request_id': 'req-d3a6988c-6037-4021-bec6-abda6f27d25f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:36.229520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] The token cache is being invalidate because project 32dee2c6dfd447488ae7077af3cca683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 07:59:36.230033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:36.231739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 07:59:36.233783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1255/2521] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 07:59:35 2022] DELETE /identity/v3/projects/32dee2c6dfd447488ae7077af3cca683 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:38.333397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:38.333397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:38.333397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:38.370770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] MFA Rules not processed for user `b3fa420e4c0049b9ae5cee7717422cda`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:38.374457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:38.470139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1267/2522] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:38.471308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:38.473091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:38.543716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:38.567190 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6988c-6037-4021-bec6-abda6f27d25f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:38.572420 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:38.573085 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:38.573367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:38.573788 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:38.575442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:38.580620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:38.673900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1256/2523] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 07:59:38 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 07:59:38.675465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:38.686585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848d555-5b0f-4ff6-89b8-8ad614b86485 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:38.723983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:38.754706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:38.755314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:38.755588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:38.755860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:38.756492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:38.758469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:38.840035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1268/2524] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 07:59:38 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 07:59:38.841217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:38.855609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136ca38-40a4-4d4c-b5ec-adf014f558e0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:38.880509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:38.881177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:38.881459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:38.881734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:38.882404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:38.884022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:38.895751 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1257/2525] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 07:59:38.900832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:38.923664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b163a-8475-4ed1-b253-075ebe650bc6 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:38.930907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:38.931846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:38.932645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:38.933225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] PATH_INFO: `/v3/projects/2b9ec53274cc4c8c8e54c3811b712626/users/002d566075f54467b5db9cb6eca80fd7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:38.950889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9ec53274cc4c8c8e54c3811b712626, user_id=002d566075f54467b5db9cb6eca80fd7, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:38.953764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:39.023697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1269/2526] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:59:38 2022] PUT /identity/v3/projects/2b9ec53274cc4c8c8e54c3811b712626/users/002d566075f54467b5db9cb6eca80fd7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:39.032974 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:39.043329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0ea3e-300f-4189-abc0-3c91c06fd91d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:39.108592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:39.109257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:39.109535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:39.109810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:39.110463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:39.112066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a449d0-9541-48c2-8060-feb3ecfab837 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:39.122678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1258/2527] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 07:59:39.126919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:39.139771 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03af9ff-166c-479f-a52e-4e8def2653d1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:39.144827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:39.148023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:39.148023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:39.148023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] PATH_INFO: `/v3/projects/2b9ec53274cc4c8c8e54c3811b712626/users/002d566075f54467b5db9cb6eca80fd7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:39.148023 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9ec53274cc4c8c8e54c3811b712626, user_id=002d566075f54467b5db9cb6eca80fd7, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:39.150812 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:39.164985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1270/2528] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 07:59:39 2022] PUT /identity/v3/projects/2b9ec53274cc4c8c8e54c3811b712626/users/002d566075f54467b5db9cb6eca80fd7/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 07:59:45.240266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ac6a-c835-4a81-baf8-e704d3ca94c6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:45.243003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ac6a-c835-4a81-baf8-e704d3ca94c6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:45.243448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188ac6a-c835-4a81-baf8-e704d3ca94c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:45.285633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-e188ac6a-c835-4a81-baf8-e704d3ca94c6 None None] MFA Rules not processed for user `002d566075f54467b5db9cb6eca80fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:45.287726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-e188ac6a-c835-4a81-baf8-e704d3ca94c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:45.427395 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1259/2529] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:45.437103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:45.440391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:45.508956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cda97-bfb4-4662-9e86-25ce67bf9d47 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:45.513352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b95e296a3e2f48599b916a0ac7be2213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:45.514113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:45.514420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:45.514670 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:45.578496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:45.582571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:45.591169 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1271/2530] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 07:59:45.882282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:45.882396 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:45.882676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:45.903199 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:45.904440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:45.958532 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:45.983459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1260/2531] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 07:59:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 07:59:45.997133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:45.998498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:46.063839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:59:46.092837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdb129-c7a6-4164-8ceb-20f0d4f86260 service glance] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:46.098356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:46.098975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:46.099255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:46.099528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:46.109910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:46.111742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68458cc2-9cdf-4b94-b08f-ba5e3c61c055 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:46.122638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1272/2532] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 07:59:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 07:59:52.279440 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:59:52.281161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:52.401778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3faa-e14b-466f-8ae1-c45abe8380b0 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:59:52.407843 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:59:52.408465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:52.408778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:52.409051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:52.410849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:52.537221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8397c-bac0-4d7d-9822-04845fd8de6c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:52.540194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8397c-bac0-4d7d-9822-04845fd8de6c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:52.540194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8397c-bac0-4d7d-9822-04845fd8de6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:52.542576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:59:52.544694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b0420-0919-4f1b-b550-e16f7fefec98 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:59:52.555913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1261/2533] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 07:59:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:59:52.566650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b5b8397c-bac0-4d7d-9822-04845fd8de6c None None] MFA Rules not processed for user `51729e82a3784d04aef89ed83f1576bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:52.567952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b5b8397c-bac0-4d7d-9822-04845fd8de6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:52.648170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1273/2534] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 07:59:58.404214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:59:58.404846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:59:58.404846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:59:58.430715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] MFA Rules not processed for user `b3fa420e4c0049b9ae5cee7717422cda`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 07:59:58.430715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:59:58.434624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1262/2535] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 07:59:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:00:02.640915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=39 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:03.331502 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b0d-fafa-4d0d-b22a-c6e98f7b649e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:03.331931 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b0d-fafa-4d0d-b22a-c6e98f7b649e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:03.332211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6b0d-fafa-4d0d-b22a-c6e98f7b649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:03.352786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9d0e6b0d-fafa-4d0d-b22a-c6e98f7b649e None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:00:03.354436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9d0e6b0d-fafa-4d0d-b22a-c6e98f7b649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:03.375607 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1274/2536] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 08:00:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:00:03.376523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:03.393871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617f698-1a7a-4a86-af70-2223f50eee48 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:03.399140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:03.399790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:03.400072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:03.400516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:03.411390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:03.413290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:03.423992 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1263/2537] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 08:00:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 08:00:05.155878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:05.156775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:05.157206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:05.211674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] MFA Rules not processed for user `d318793a5fee43bdb1a043b30396884b`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:00:05.222142 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:05.279236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1275/2538] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:00:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:00:13.419337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=65 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:15.271317 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=28 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:26.106565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:26.115917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d39968-60dd-4687-802a-1a108a8a1ed1 service swift] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:26.119993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:26.120604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:26.120906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:26.121174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:26.123498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:26.196496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:26.198637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43ee72-99d5-4cbb-a3eb-1354bcc1049c service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:26.212297 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1264/2539] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:27.503345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:27.506486 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:27.517386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50b6d9-cffd-4524-ad73-bf65a859e433 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:27.521595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:27.522399 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:27.522750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:27.523197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:27.525567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:27.539790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:27.541240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:27.552758 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1276/2540] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:36.206462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=34 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:37.547981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:40.181389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:40.181893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:40.181989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:40.202401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] MFA Rules not processed for user `5a8de6b6200840dbb9c4d7a3b728f9c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:00:40.203600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:40.258320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1265/2541] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:00:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:00:42.388890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:42.399279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9633-dd69-4bb0-a204-1a1f4a5ce4d6 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:42.408861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:42.409712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:42.409992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:42.410264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:42.422087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:42.423795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:42.435375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1277/2542] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:48.168454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:48.170550 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:48.257273 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02e1bf-399d-4f7b-8d7a-5a6974013003 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:48.264925 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:48.265558 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:48.265906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:48.266185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:48.268185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:48.287122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:48.288575 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:48.301482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1266/2543] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:52.430101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:00:54.058283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:00:54.066934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32980c44-9c8d-4298-9cba-a1bf6c988a47 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:00:54.070596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:00:54.071181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:00:54.071461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:00:54.071735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:00:54.073514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:00:54.083140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:00:54.084685 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:00:54.093481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1278/2544] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:00:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:00:58.295947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=51 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:04.089195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:04.415531 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:04.417877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:04.437063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb3184e-750d-4e58-907e-6bc9e364bf3e service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:04.443664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:04.444325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:04.444690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:04.445031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:04.447608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:04.463303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:04.465082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:04.477677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1267/2545] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:14.289855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:14.292247 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:14.310986 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02543122-5d41-47b9-aed9-abe9b2139e92 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:14.318503 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:14.319279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:14.319662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:14.319960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:14.321871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:14.337746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:14.339781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:14.350404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1279/2546] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:14.472597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:14.857498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:14.869026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56507c-fdde-4b52-829a-bf03afb6a9bb service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:14.876342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:14.877342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:14.877799 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:14.878246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:14.880340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:14.894116 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:14.895879 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:14.906709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1268/2547] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:16.885672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:16.895294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15689b0b-5208-4eea-bf05-7bca412ace19 service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:16.899054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:16.899792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:16.899909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:16.900180 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:16.901937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:16.967195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:16.969318 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:16.979912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1280/2548] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:20.880352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:20.884739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:20.952907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c55b4-6075-47f8-a72a-ec39eb0763b0 service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:20.958813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:20.959426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:20.959706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:20.959980 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:20.961748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:20.974083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:20.975686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:20.986153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1269/2549] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:21.006698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:21.008426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:21.023657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4918962-a377-4ffd-a957-48395742931f service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:21.029105 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:21.030196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:21.030641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:21.030908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] PATH_INFO: `/v3/users/51729e82a3784d04aef89ed83f1576bd` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:21.032716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51729e82a3784d04aef89ed83f1576bd)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:21.035133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:21.218614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51729e82a3784d04aef89ed83f1576bd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.219294 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51729e82a3784d04aef89ed83f1576bd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.228319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51729e82a3784d04aef89ed83f1576bd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.288149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51729e82a3784d04aef89ed83f1576bd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.306254 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-387927ee-7f77-4d11-a44d-217592b7be43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51729e82a3784d04aef89ed83f1576bd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.314855 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1281/2550] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:21 2022] DELETE /identity/v3/users/51729e82a3784d04aef89ed83f1576bd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:21.576350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:21.588886 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742397-241a-4eb9-bfa1-a8d0db53446e service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:21.653246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:21.653885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:21.654168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:21.654448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] PATH_INFO: `/v3/projects/e52495240cf04548a21b411c32bbe1e1` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:21.655576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52495240cf04548a21b411c32bbe1e1)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:21.658338 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:21.741258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52495240cf04548a21b411c32bbe1e1', 'request_id': 'req-39059f5f-8115-47fe-af40-63815e57835e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.747347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52495240cf04548a21b411c32bbe1e1', 'request_id': 'req-39059f5f-8115-47fe-af40-63815e57835e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.755068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52495240cf04548a21b411c32bbe1e1', 'request_id': 'req-39059f5f-8115-47fe-af40-63815e57835e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.830961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] The token cache is being invalidate because project e52495240cf04548a21b411c32bbe1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:21.831255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.831896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-39059f5f-8115-47fe-af40-63815e57835e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:21.833518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1270/2551] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:21 2022] DELETE /identity/v3/projects/e52495240cf04548a21b411c32bbe1e1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:21.838571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:21.838857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:21.839130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:21.859738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:21.861011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:21.910222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:21.922807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1282/2552] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:21.930559 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:21.931042 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:21.931132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:21.945768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:21.951162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:21.953936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:21.961339 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1271/2553] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:21.965583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:21.976130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8229e8-422f-426d-ad27-b171254934c1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:21.981465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:21.982072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:21.982348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:21.982616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:21.983291 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:01:21.983591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:21.985242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:21.997287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1283/2554] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:01:22.001772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.011700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1133d9-cded-4276-897a-619d9011cfbb None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.016866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.017463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.017742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.018009 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.018676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.020637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.150995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1272/2555] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:22 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:01:22.151963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.163035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0a67-a239-4140-ac49-209abd240dbc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.228728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.229436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.229785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.230130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.230858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.232904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.321593 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1284/2556] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:22 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:22.322778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.339812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dc88-a0e3-45d7-8982-83c7b7b9c2c6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.344940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.345556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.346162 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.346438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.347093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.348745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.362355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1273/2557] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:22.363546 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.374552 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e19-c561-471d-9d39-dc0cd362e0b8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.384792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.389621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.389621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.389621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.389621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.389621 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7d5c6-a02f-415c-b38a-7e1107299950 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.532273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1285/2558] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:22.533376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.543661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f8463-0689-4bea-9b77-944a860e26b1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.548347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.549022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.549304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.549580 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] PATH_INFO: `/v3/projects/f4e18b3870e0429dbe00345d4fc9d462/users/561edc3bedd64b70b926530507519dcf/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.564746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e18b3870e0429dbe00345d4fc9d462, user_id=561edc3bedd64b70b926530507519dcf, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.567442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.699778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1274/2559] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:22 2022] PUT /identity/v3/projects/f4e18b3870e0429dbe00345d4fc9d462/users/561edc3bedd64b70b926530507519dcf/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:22.711097 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.711321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.711595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.741749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] MFA Rules not processed for user `561edc3bedd64b70b926530507519dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:22.743492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:22.815887 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1286/2560] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:01:22.826767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:22.828745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:22.904620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abfb3-76d5-4858-a09c-bd0fb46fe120 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:22.910471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:22.911080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:22.911357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:22.911628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:22.919633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:22.921253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3d738-10c6-480d-810a-096c11aa9e51 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:22.930942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1275/2561] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:01:26.097474 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:26.143538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:26.157205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd0e6-a44d-411a-b735-d73d0a356a47 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:26.161716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:26.162340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:26.162616 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:26.162884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] PATH_INFO: `/v3/users/561edc3bedd64b70b926530507519dcf` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:26.163993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561edc3bedd64b70b926530507519dcf)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:26.165936 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:26.324268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561edc3bedd64b70b926530507519dcf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:26.324964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561edc3bedd64b70b926530507519dcf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:26.332615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561edc3bedd64b70b926530507519dcf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:26.402754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561edc3bedd64b70b926530507519dcf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:26.417831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561edc3bedd64b70b926530507519dcf'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:26.424306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1287/2562] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:26 2022] DELETE /identity/v3/users/561edc3bedd64b70b926530507519dcf => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:26.429105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:26.429631 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:26.429732 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:26.448720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:26.450253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:26.510615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1276/2563] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:26.520935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:26.522436 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:26.583974 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a88c-cc29-44ef-b71e-70f17a8fb9d0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:26.587813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:26.588501 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:26.588808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:26.589085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:26.598727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:26.600183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc1c7-9458-4a5d-ad2c-5ba603636ff8 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:26.610056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1288/2564] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:27.089421 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.090859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:27.103870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df147e6-7898-44b9-853f-1238676b15f7 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.108471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.109648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.109967 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.110207 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] PATH_INFO: `/v3/projects/f4e18b3870e0429dbe00345d4fc9d462` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.111327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e18b3870e0429dbe00345d4fc9d462)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.113256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.140623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:27.235388 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e18b3870e0429dbe00345d4fc9d462', 'request_id': 'req-b1078435-2fa5-46ee-a026-ff0d96712465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:27.240939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e18b3870e0429dbe00345d4fc9d462', 'request_id': 'req-b1078435-2fa5-46ee-a026-ff0d96712465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:27.247177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e18b3870e0429dbe00345d4fc9d462', 'request_id': 'req-b1078435-2fa5-46ee-a026-ff0d96712465'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:27.338766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] The token cache is being invalidate because project f4e18b3870e0429dbe00345d4fc9d462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:27.339056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:27.339775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b1078435-2fa5-46ee-a026-ff0d96712465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:27.341249 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1277/2565] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:27 2022] DELETE /identity/v3/projects/f4e18b3870e0429dbe00345d4fc9d462 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:27.346157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.346632 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.346740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.366427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:27.368492 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:27.428357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1289/2566] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:27.444019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.444551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.444551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.463924 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:27.465151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:27.473465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1278/2567] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:27.477924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.488588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c248cd2-a4f6-453f-abf2-103b3dad738f None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.493264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.500478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.503802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:27.508736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1290/2568] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:01:27.513088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.522279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac35e37-60a7-4451-8391-ceb66efb32d5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.527143 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.540293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.540293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.540293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.540293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.540293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.647232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1279/2569] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:27 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:27.651443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.662048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058cc4-7955-402d-9634-8a6823c9de1c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.720323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.721188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.721409 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.721681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.722298 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.724204 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.840524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1291/2570] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:27 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:27.845465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.855894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e3d566-6dff-4977-9a28-941c08f604d8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.860563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.861191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.861501 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.861801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.862444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.864033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.874145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1280/2571] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:27.878117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.888114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49c706-f727-459d-ac49-31e18c6e19ea admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.893160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.893792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.894067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.894337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.894966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.896570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3730129-394e-49af-bbd4-687c75dc99a8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:27.908035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1292/2572] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:27.911354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:27.921286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeeb8c-2918-4241-8bf2-d8ff6000407a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:27.926193 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:27.926847 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:27.927127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:27.927404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] PATH_INFO: `/v3/projects/437ada9eacc94b31b5087c30ad234dcd/users/ca92a52d207f491599f2f09c79729321/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:27.943019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=437ada9eacc94b31b5087c30ad234dcd, user_id=ca92a52d207f491599f2f09c79729321, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:27.945710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.010876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1281/2573] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:27 2022] PUT /identity/v3/projects/437ada9eacc94b31b5087c30ad234dcd/users/ca92a52d207f491599f2f09c79729321/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:28.022018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.022305 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.022579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.050778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] MFA Rules not processed for user `ca92a52d207f491599f2f09c79729321`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:28.051978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:28.117106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1293/2574] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:28.121651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.186966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146c64-655c-4251-877b-fe3633abacbd admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.192432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.193083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.193372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.193648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.194294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.196246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.288168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1282/2575] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:28 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:28.292299 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.303480 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e157-668b-4544-ac77-cf448faed0d4 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.358473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.359210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.360032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.360301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.360941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.362831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.373337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:28.482462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1294/2576] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:28 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:28.487062 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.497591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8095-9ecb-48f6-ba1f-fb4619d7f705 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.502576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.503463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.503938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.504293 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.505332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.507912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.526838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1283/2577] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:28.532047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.544982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4320-87dd-40de-b510-da70fa0109c2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.550626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.551312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.551588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.551858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] PATH_INFO: `/v3/projects/1a616243ec5d484f85ce90d282f5bab7/users/0715c95fa47a4eeb91c2cf985709b139/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.567768 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a616243ec5d484f85ce90d282f5bab7, user_id=0715c95fa47a4eeb91c2cf985709b139, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.570581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.642667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1295/2578] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:28 2022] PUT /identity/v3/projects/1a616243ec5d484f85ce90d282f5bab7/users/0715c95fa47a4eeb91c2cf985709b139/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:28.646944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.657983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cb359-b0d0-44f0-8e8e-04300518f855 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.675978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:28.714031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.714640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.714919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.715216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.715894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.717561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd0b39-6cbd-4039-8553-860086f1915a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.728411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1284/2579] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:28.732712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.743173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93333c-54df-4ab1-aaf5-c4f553915db1 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.747994 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.748642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.748951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.749217 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] PATH_INFO: `/v3/projects/1a616243ec5d484f85ce90d282f5bab7/users/0715c95fa47a4eeb91c2cf985709b139/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.751198 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a616243ec5d484f85ce90d282f5bab7, user_id=0715c95fa47a4eeb91c2cf985709b139, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.753857 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.764475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1296/2580] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:28 2022] PUT /identity/v3/projects/1a616243ec5d484f85ce90d282f5bab7/users/0715c95fa47a4eeb91c2cf985709b139/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:28.777441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.777728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.778002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.807369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] MFA Rules not processed for user `0715c95fa47a4eeb91c2cf985709b139`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:28.808596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:28.881040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1285/2581] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:28.896418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:28.898194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:28.957338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e36414-7626-4829-965c-d68f796bcfea admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:28.960954 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:28.961559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:28.961836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:28.962104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:28.971561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:28.973018 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:28.982212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1297/2582] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:29.211819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:29.221413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2f7a-c523-44b4-b69c-5f29af300800 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:29.225036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:29.225633 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:29.225912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:29.226185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:29.283877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:29.285369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69063f-40cb-48c1-8cb9-4b6bca2a4030 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:29.294458 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1286/2583] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:36.033645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:36.094796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eea4c6-584d-4975-afe9-7046bf44cdfe service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:36.099266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:36.099885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:36.100163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:36.100434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] PATH_INFO: `/v3/users/ca92a52d207f491599f2f09c79729321` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:36.101618 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca92a52d207f491599f2f09c79729321)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:36.103531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:36.331052 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca92a52d207f491599f2f09c79729321'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:36.331706 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca92a52d207f491599f2f09c79729321'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:36.338872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca92a52d207f491599f2f09c79729321'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:36.408242 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca92a52d207f491599f2f09c79729321'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:36.409684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:36.422273 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca92a52d207f491599f2f09c79729321'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:36.429505 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1298/2584] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:36 2022] DELETE /identity/v3/users/ca92a52d207f491599f2f09c79729321 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:36.433834 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:36.434347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:36.434483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:36.453926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:36.455140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:36.495312 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:36.522721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1287/2585] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:36.530036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:36.531721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:36.591114 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5028c20-8a66-4901-aac1-6de57d2242e0 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:36.594678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:36.595278 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:36.595557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:36.595827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:36.605500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:36.606922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:36.616843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1299/2586] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:37.036754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:37.038218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:37.050184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400efd6-6b6f-4e9e-97ad-8a7c2ccb06c5 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:37.054919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:37.055551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:37.055825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:37.056096 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] PATH_INFO: `/v3/projects/437ada9eacc94b31b5087c30ad234dcd` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:37.057238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437ada9eacc94b31b5087c30ad234dcd)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:37.059235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:37.176866 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437ada9eacc94b31b5087c30ad234dcd', 'request_id': 'req-7d14281e-0f8d-4397-b0e6-7420853f61f7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.182302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437ada9eacc94b31b5087c30ad234dcd', 'request_id': 'req-7d14281e-0f8d-4397-b0e6-7420853f61f7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.188444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437ada9eacc94b31b5087c30ad234dcd', 'request_id': 'req-7d14281e-0f8d-4397-b0e6-7420853f61f7'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.264376 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] The token cache is being invalidate because project 437ada9eacc94b31b5087c30ad234dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:37.264758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.265397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.266923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1288/2587] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:37 2022] DELETE /identity/v3/projects/437ada9eacc94b31b5087c30ad234dcd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:37.271783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:37.281448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ba73f-1873-4147-a9e8-1427c4a3c581 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:37.337738 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:37.338364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:37.338640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:37.338909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] PATH_INFO: `/v3/users/0715c95fa47a4eeb91c2cf985709b139` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:37.340045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0715c95fa47a4eeb91c2cf985709b139)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:37.341983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:37.558992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0715c95fa47a4eeb91c2cf985709b139'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.559661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0715c95fa47a4eeb91c2cf985709b139'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.566653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0715c95fa47a4eeb91c2cf985709b139'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.633982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0715c95fa47a4eeb91c2cf985709b139'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.646788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cc122255-b03a-4ef3-ab6c-3fced1cf3c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0715c95fa47a4eeb91c2cf985709b139'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:37.652839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1300/2588] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:37 2022] DELETE /identity/v3/users/0715c95fa47a4eeb91c2cf985709b139 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:38.087644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.089206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:38.104565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14281e-0f8d-4397-b0e6-7420853f61f7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.164152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.164845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.165131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.165409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] PATH_INFO: `/v3/projects/1a616243ec5d484f85ce90d282f5bab7` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.166739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a616243ec5d484f85ce90d282f5bab7)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.168703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.294808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a616243ec5d484f85ce90d282f5bab7', 'request_id': 'req-95fe35b3-de48-47fe-a286-9b417c15d6b8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:38.300363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a616243ec5d484f85ce90d282f5bab7', 'request_id': 'req-95fe35b3-de48-47fe-a286-9b417c15d6b8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:38.306649 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a616243ec5d484f85ce90d282f5bab7', 'request_id': 'req-95fe35b3-de48-47fe-a286-9b417c15d6b8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:38.310326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:38.365621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] The token cache is being invalidate because project 1a616243ec5d484f85ce90d282f5bab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:38.365829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:38.366455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-95fe35b3-de48-47fe-a286-9b417c15d6b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:38.367955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1289/2589] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:38 2022] DELETE /identity/v3/projects/1a616243ec5d484f85ce90d282f5bab7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:38.373313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.373816 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.373911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.381561 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:38.393521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:38.395285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:38.454144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1301/2590] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:38.462442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.462728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.463031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.483343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:38.484606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:38.493214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1290/2591] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:38.497716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.507374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c0e32-a96f-4edb-9a09-caaae78373ad None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.511896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.512490 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.512807 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.513082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.513727 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:01:38.514055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.515647 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.528048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1302/2592] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:01:38.532609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.542790 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45b67d-467e-42a2-8aae-3656c45cf16b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.547701 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.548310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.548591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.548911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.549591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.551714 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.666295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1291/2593] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:38 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:38.670356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.680418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe3c64-07b1-4303-953b-f4444d3a4424 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.736673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.737292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.737566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.737835 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.738461 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.740377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.810287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1303/2594] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:38 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:38.814489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.826572 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46e249-1240-46c9-b08b-46ccd400a265 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.832813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.836814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.836814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.836814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.836814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.836814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.847988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1292/2595] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:38.852047 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.862604 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d40acb-1d36-4f85-b2b2-1426c680086e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.867792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.868442 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.868747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.869029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.869671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.871268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed1818-1a7e-4593-b3f8-ac165e79987b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:38.887215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1304/2596] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:38.888502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:38.899593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429b2f9-a8b3-437c-b979-d06e2505b428 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:38.905167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:38.907700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:38.907700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:38.907700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] PATH_INFO: `/v3/projects/cccaa81f6b4d42b0bac69c1100e7ba26/users/e492801249654fccb89a62883d3d11e8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:38.923100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccaa81f6b4d42b0bac69c1100e7ba26, user_id=e492801249654fccb89a62883d3d11e8, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:38.925784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.065491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1293/2597] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:38 2022] PUT /identity/v3/projects/cccaa81f6b4d42b0bac69c1100e7ba26/users/e492801249654fccb89a62883d3d11e8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:39.077613 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.077897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.078168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.107100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] MFA Rules not processed for user `e492801249654fccb89a62883d3d11e8`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:39.108520 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:39.169246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:39.174901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1305/2598] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:39.179945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.244585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96578b65-94a1-4ebe-939f-b5a0d41b0b6e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.249546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.250184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.250465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.250738 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.251373 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.253347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.340964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1294/2599] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:39 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:39.344798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.356043 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a8ce1-a3d8-4433-8d5b-5a4a56732c6e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.414535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.415146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.415432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.415707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.416345 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.418292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.543117 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1306/2600] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:39 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:39.544639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.555956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9a6d-16db-4956-a315-2f0faab832c7 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.560604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.561232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.561513 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.561787 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.562459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.564150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.575216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1295/2601] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:39.579545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.589601 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da766966-7df1-4316-80af-c5feb87bb7da admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.594351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.595027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.595307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.595579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] PATH_INFO: `/v3/projects/905eb4eefd4946738ff616eda2c3b053/users/baed5c9c1beb49cba7643e03e4c036c2/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.611054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=905eb4eefd4946738ff616eda2c3b053, user_id=baed5c9c1beb49cba7643e03e4c036c2, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.614373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.743895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1307/2602] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:39 2022] PUT /identity/v3/projects/905eb4eefd4946738ff616eda2c3b053/users/baed5c9c1beb49cba7643e03e4c036c2/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:39.748061 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.758632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e540ea-3daa-47a8-97f8-fdbcc1aad1f3 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.818882 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.819496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.819772 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.820045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.820710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.822276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a034f-9491-4ee7-a5a4-91d580efdaf2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.836132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1296/2603] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:39.840293 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.851608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239c037-5608-482a-b2b5-b9d455f67c8e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:39.856301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:39.856988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.857292 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.862384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] PATH_INFO: `/v3/projects/905eb4eefd4946738ff616eda2c3b053/users/baed5c9c1beb49cba7643e03e4c036c2/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.862384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905eb4eefd4946738ff616eda2c3b053, user_id=baed5c9c1beb49cba7643e03e4c036c2, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:39.862384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:39.872268 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1308/2604] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:39 2022] PUT /identity/v3/projects/905eb4eefd4946738ff616eda2c3b053/users/baed5c9c1beb49cba7643e03e4c036c2/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:39.882817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db9c-481b-4db8-9572-e3d5e3066530 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:39.883100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db9c-481b-4db8-9572-e3d5e3066530 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:39.883374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1db9c-481b-4db8-9572-e3d5e3066530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:39.890568 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:39.912975 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f4b1db9c-481b-4db8-9572-e3d5e3066530 None None] MFA Rules not processed for user `baed5c9c1beb49cba7643e03e4c036c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:39.914288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f4b1db9c-481b-4db8-9572-e3d5e3066530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:39.984817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1297/2605] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:39.995138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:39.997126 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:40.059697 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48f9dd-a144-4356-8974-96bdbb09d8b9 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:40.063746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:40.064365 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:40.064740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:40.065020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:40.074753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:40.076183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:40.086759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1309/2606] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:40.862708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:40.864302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:40.865194 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:40.902837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:40.904911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:40.971164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1298/2607] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 08:01:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:01:41.013197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:41.014619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:41.070444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:41.087907 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63ff8d-823c-453b-8d13-2deeaecb1ee0 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:41.095614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:41.095614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:41.095614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:41.095614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:41.104325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:41.105473 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ae3c6-4bdc-47a8-81ce-1cef1a7bb28f service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:41.114540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1310/2608] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 08:01:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:01:41.273825 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:41.391342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0355f51-3584-418b-bf6b-be8184322bf3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:41.392645 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:41.394489 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:41.395433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:41.395707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] PATH_INFO: `/v3/users/e492801249654fccb89a62883d3d11e8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:41.397107 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e492801249654fccb89a62883d3d11e8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:41.399345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:41.657294 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:41.676444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e492801249654fccb89a62883d3d11e8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:41.684784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e492801249654fccb89a62883d3d11e8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:41.685535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e492801249654fccb89a62883d3d11e8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:41.686175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e492801249654fccb89a62883d3d11e8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:41.779464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e492801249654fccb89a62883d3d11e8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:41.802957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1299/2609] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:41 2022] DELETE /identity/v3/users/e492801249654fccb89a62883d3d11e8 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:41.807763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:41.808089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:41.808391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:41.830014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:41.831369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:41.940803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1311/2610] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:41.952898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:41.954481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:42.039140 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c894735-4953-4ea4-94ad-49adbab19791 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:42.043124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:42.043748 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:42.044031 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:42.044307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:42.060826 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:42.062515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:42.083264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1300/2611] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:42.675010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:42.720596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:42.786067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:42.786067 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:42.802480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805e13-ba3c-43a9-920e-1feb6aa5190b service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:42.819604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:42.821037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:42.821494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:42.822931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] PATH_INFO: `/v3/projects/cccaa81f6b4d42b0bac69c1100e7ba26` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:42.825630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccaa81f6b4d42b0bac69c1100e7ba26)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:42.829086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:42.916119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c87a9-795a-4f94-af4b-a92edec38a3e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:42.922545 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:42.923160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:42.923439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:42.923802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:42.925612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:42.939853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:42.942184 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:42.953691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1312/2612] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:01:43.174054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccaa81f6b4d42b0bac69c1100e7ba26', 'request_id': 'req-61896e63-768a-49a9-97c8-e8f95ef20065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.181329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccaa81f6b4d42b0bac69c1100e7ba26', 'request_id': 'req-61896e63-768a-49a9-97c8-e8f95ef20065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.188922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccaa81f6b4d42b0bac69c1100e7ba26', 'request_id': 'req-61896e63-768a-49a9-97c8-e8f95ef20065'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.263837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] The token cache is being invalidate because project cccaa81f6b4d42b0bac69c1100e7ba26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:43.264131 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.266511 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.268303 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1301/2613] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:42 2022] DELETE /identity/v3/projects/cccaa81f6b4d42b0bac69c1100e7ba26 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:43.272569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:43.284774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e04a09-754d-4dfa-b5de-cc130ee6774b service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:43.356324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:43.357011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:43.357281 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:43.357549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] PATH_INFO: `/v3/users/baed5c9c1beb49cba7643e03e4c036c2` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:43.358688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] RBAC: Authorizing `identity:delete_user(user_id=baed5c9c1beb49cba7643e03e4c036c2)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:43.360700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:43.377366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=100 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:43.573311 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baed5c9c1beb49cba7643e03e4c036c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.574053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baed5c9c1beb49cba7643e03e4c036c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.584360 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baed5c9c1beb49cba7643e03e4c036c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.674752 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baed5c9c1beb49cba7643e03e4c036c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.688901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-186f16fc-3480-4ae0-837a-37822cb5a56c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baed5c9c1beb49cba7643e03e4c036c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:43.698663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1313/2614] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:43 2022] DELETE /identity/v3/users/baed5c9c1beb49cba7643e03e4c036c2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:44.662901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:44.664482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:44.693341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61896e63-768a-49a9-97c8-e8f95ef20065 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:44.741011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:44.768996 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:44.769893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:44.770340 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:44.770647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] PATH_INFO: `/v3/projects/905eb4eefd4946738ff616eda2c3b053` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:44.771906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905eb4eefd4946738ff616eda2c3b053)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:44.780680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:44.981488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905eb4eefd4946738ff616eda2c3b053', 'request_id': 'req-cf1fac4a-116c-4221-b2dd-5e8a34866764'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:44.988776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905eb4eefd4946738ff616eda2c3b053', 'request_id': 'req-cf1fac4a-116c-4221-b2dd-5e8a34866764'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:44.996231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905eb4eefd4946738ff616eda2c3b053', 'request_id': 'req-cf1fac4a-116c-4221-b2dd-5e8a34866764'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:45.079425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] The token cache is being invalidate because project 905eb4eefd4946738ff616eda2c3b053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:45.079723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:45.080399 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-cf1fac4a-116c-4221-b2dd-5e8a34866764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:45.082509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1302/2615] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:44 2022] DELETE /identity/v3/projects/905eb4eefd4946738ff616eda2c3b053 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:45.088186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.088865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.089050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.112416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:45.113933 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:45.194938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1314/2616] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:45.220508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.220958 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.221466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.247361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:45.247641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:45.260694 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1303/2617] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:45.263999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.275882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745038e-09eb-4a29-a83a-365de4f500e7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.280553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.281194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.281858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.282132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.282778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:01:45.283195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.284893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.306747 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1315/2618] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:01:45.313744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.333864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ee49b-8dbb-4b92-8b69-17205f545210 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.349086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.349841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.350178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.350508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.351200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.353184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.464211 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1304/2619] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:45 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:45.469160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.479498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0c6e6-f6b5-4444-a068-92c97b6ffaa5 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.551786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.552495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.552897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.553233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.553935 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.555910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.748995 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1316/2620] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:45 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:45.750322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.762829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f471fc1-5c8e-41d0-8459-62380d9e681d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.767832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.773017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.773599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.773938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.774693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.777754 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.794754 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.802715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1305/2621] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:45.802881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:45.806836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28deec4-7153-41a5-9ccb-5bea09cf320e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.811698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.812369 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.812672 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.812956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.813641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.815663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7377f5-fc1d-4dda-8105-b82179b21a16 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.828310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1317/2622] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:45.833006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:45.845957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22f911-6bff-421a-b879-ff1bfbe80e0c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:45.851969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:45.852639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.852962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.853238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] PATH_INFO: `/v3/projects/759ac60f1fe3475fa7840a56643764b3/users/8c4eb721182b4821aa7011f5063b63e5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:45.874254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=759ac60f1fe3475fa7840a56643764b3, user_id=8c4eb721182b4821aa7011f5063b63e5, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:45.877309 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:45.975248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1306/2623] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:45 2022] PUT /identity/v3/projects/759ac60f1fe3475fa7840a56643764b3/users/8c4eb721182b4821aa7011f5063b63e5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:45.987174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:45.987610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:45.987932 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.023233 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] MFA Rules not processed for user `8c4eb721182b4821aa7011f5063b63e5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:46.024599 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:46.101662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1318/2624] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:46.105846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.175299 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc5ca8-2f1f-409d-a040-7a7c1ac13a4e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.181192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.181800 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.182073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.182351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.182978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.184920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.197956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:46.301894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1307/2625] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:01:46 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:46.304493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.316201 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660ee53-40f0-4184-a034-5ad85fb9c070 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.389757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.390688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.391190 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.391620 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.392455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.394967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.486543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1319/2626] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:01:46 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:01:46.490663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.507497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832277a2-b799-4c36-a29f-dfc1cf430402 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.514007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.514903 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.515323 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.515683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.516415 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.518157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.542850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1308/2627] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:46.548585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.561793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f428-12f3-44d9-97d6-4d7021c54f5d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.570238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.571129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.571500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.572588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] PATH_INFO: `/v3/projects/38b743bb579640e3ae8d26e73e6782fb/users/910e0557589a438f8f43681b491a03f9/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.590853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b743bb579640e3ae8d26e73e6782fb, user_id=910e0557589a438f8f43681b491a03f9, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.593660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.682846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1320/2628] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:46 2022] PUT /identity/v3/projects/38b743bb579640e3ae8d26e73e6782fb/users/910e0557589a438f8f43681b491a03f9/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:01:46.684158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.695087 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025c55-c0ba-46e1-aa53-5651ab9fafed admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.781831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.782545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.782830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.783161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.784033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.786672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98720746-cb78-4759-aece-5379af6b984e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.805329 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1309/2629] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:01:46.806304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:46.819543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dacbf-9798-42ad-b199-c372462f012a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:46.824804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:46.825675 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.826147 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.826530 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] PATH_INFO: `/v3/projects/38b743bb579640e3ae8d26e73e6782fb/users/910e0557589a438f8f43681b491a03f9/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.828788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b743bb579640e3ae8d26e73e6782fb, user_id=910e0557589a438f8f43681b491a03f9, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:46.836677 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:46.850148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1321/2630] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:01:46 2022] PUT /identity/v3/projects/38b743bb579640e3ae8d26e73e6782fb/users/910e0557589a438f8f43681b491a03f9/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:46.861740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:46.862069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:46.862350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:46.897100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] MFA Rules not processed for user `910e0557589a438f8f43681b491a03f9`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:46.898909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:46.989928 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1310/2631] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:47.001456 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:47.004302 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:47.050483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:47.072393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a915-f988-4b49-b721-96c6fef4a959 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:47.076866 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:47.077567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:47.078040 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:47.078314 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:47.147800 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:47.149232 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:47.160257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1322/2632] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:51.203976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:51.205921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:51.305150 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:51.312385 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02260c-7579-4655-a963-9ffb289a3f24 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:51.318342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:51.319060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:51.319454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:51.319762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:51.321708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:51.336979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:51.339028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:51.364758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1311/2633] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:51.365154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:51.367161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:51.469527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0c6fb-d4a3-4bd3-8499-2a666ef27067 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:51.479603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:51.480395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:51.480763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:51.481110 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] PATH_INFO: `/v3/users/5a8de6b6200840dbb9c4d7a3b728f9c2` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:51.482689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8de6b6200840dbb9c4d7a3b728f9c2)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:51.484976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:51.705257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8de6b6200840dbb9c4d7a3b728f9c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:51.706332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8de6b6200840dbb9c4d7a3b728f9c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:51.716500 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8de6b6200840dbb9c4d7a3b728f9c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:51.858913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8de6b6200840dbb9c4d7a3b728f9c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:51.873524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8de6b6200840dbb9c4d7a3b728f9c2'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:51.880284 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1323/2634] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:51 2022] DELETE /identity/v3/users/5a8de6b6200840dbb9c4d7a3b728f9c2 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:52.181499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:52.192168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacd51a-2bf4-48c0-804a-0915b4db8c19 service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:52.250380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:52.260459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:52.260459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:52.260459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] PATH_INFO: `/v3/projects/c534e991899e457c8b41e1dbc12a6670` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:52.260459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c534e991899e457c8b41e1dbc12a6670)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:52.260459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:52.385576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c534e991899e457c8b41e1dbc12a6670', 'request_id': 'req-862a72d1-71af-4fd6-9759-85cd744ac37c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:52.391357 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c534e991899e457c8b41e1dbc12a6670', 'request_id': 'req-862a72d1-71af-4fd6-9759-85cd744ac37c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:52.397700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c534e991899e457c8b41e1dbc12a6670', 'request_id': 'req-862a72d1-71af-4fd6-9759-85cd744ac37c'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:52.480074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] The token cache is being invalidate because project c534e991899e457c8b41e1dbc12a6670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:01:52.480316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:52.480957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:52.482469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1312/2635] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:01:52 2022] DELETE /identity/v3/projects/c534e991899e457c8b41e1dbc12a6670 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:53.182394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:53.183843 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:53.196553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:53.244262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04337f16-4909-44e9-a004-8965a488d6a2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:53.293624 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:53.294155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:53.294425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:53.294693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:53.296457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:53.367791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:53.369382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:53.385674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1324/2636] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:58.988412 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:58.989868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:59.063022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a72d1-71af-4fd6-9759-85cd744ac37c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:59.068672 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:59.069280 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:59.069563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:59.069841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:59.071586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:59.083724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:59.085346 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b51eb-e617-4ac1-b564-fc6d644f60d7 service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:59.095132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1313/2637] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:01:59.269197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:59.269775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:59.333336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff699d-634c-4826-9a83-231817690754 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:59.337781 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:59.338386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:59.338661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:59.338945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] PATH_INFO: `/v3/users/8c4eb721182b4821aa7011f5063b63e5` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:59.340028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4eb721182b4821aa7011f5063b63e5)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:59.341964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:59.357927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:59.513981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4eb721182b4821aa7011f5063b63e5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:59.514762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4eb721182b4821aa7011f5063b63e5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:59.522987 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4eb721182b4821aa7011f5063b63e5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:59.670761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4eb721182b4821aa7011f5063b63e5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:59.684840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4eb721182b4821aa7011f5063b63e5'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:01:59.691356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1325/2638] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:01:59 2022] DELETE /identity/v3/users/8c4eb721182b4821aa7011f5063b63e5 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:01:59.696905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:59.697195 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:59.697468 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:59.717841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:01:59.719076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:59.723705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:01:59.789425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1314/2639] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:01:59.799138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:01:59.801218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:01:59.882362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b5810-2cd5-4ee2-b2ea-e4b3c040da9f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:01:59.886020 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:01:59.886654 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:01:59.886946 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:01:59.887213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:01:59.897104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:01:59.898533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:01:59.908390 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1326/2640] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:00.329177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:00.330705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:00.346540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d739-20e4-40c2-9d47-962b00b15c76 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:00.353304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:00.353945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:00.354319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:00.354482 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] PATH_INFO: `/v3/projects/759ac60f1fe3475fa7840a56643764b3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:00.355609 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=759ac60f1fe3475fa7840a56643764b3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:00.357671 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:00.484227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759ac60f1fe3475fa7840a56643764b3', 'request_id': 'req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.490457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759ac60f1fe3475fa7840a56643764b3', 'request_id': 'req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.497929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759ac60f1fe3475fa7840a56643764b3', 'request_id': 'req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.573457 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] The token cache is being invalidate because project 759ac60f1fe3475fa7840a56643764b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:00.573680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.574351 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-4a3e6489-b4bd-43db-bb57-9bd66bf003ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.575833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1315/2641] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:00 2022] DELETE /identity/v3/projects/759ac60f1fe3475fa7840a56643764b3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:00.580014 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:00.590444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f0095-3ed8-4f65-9242-1f766f116c7f service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:00.647902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:00.648537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:00.648848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:00.651356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] PATH_INFO: `/v3/users/910e0557589a438f8f43681b491a03f9` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:00.651356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910e0557589a438f8f43681b491a03f9)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:00.652157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:00.827393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:00.834160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910e0557589a438f8f43681b491a03f9'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.839382 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910e0557589a438f8f43681b491a03f9'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.842094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910e0557589a438f8f43681b491a03f9'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.916787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910e0557589a438f8f43681b491a03f9'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.934450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910e0557589a438f8f43681b491a03f9'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:00.942144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1327/2642] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:00 2022] DELETE /identity/v3/users/910e0557589a438f8f43681b491a03f9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:02:01.176250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.176542 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.176848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.197483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:01.198761 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:01.258398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1316/2643] 10.222.0.72 () {58 vars in 1009 bytes} [Wed Aug 24 08:02:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:01.270408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:01.271832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:01.342213 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc68dee-3ca8-4c1c-817a-86a6acf08379 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:01.347914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:01.348515 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.348828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.349100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.357296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:01.358914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fe4a0-8468-4d07-9e7c-256efccb1ada service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:01.367570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1328/2644] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 08:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:02:01.390341 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:01.392044 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:01.438725 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:01.462578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004838a-3f6d-4363-9b6a-7e058dddede6 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:01.467095 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:01.467726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.468070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.468345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] PATH_INFO: `/v3/projects/38b743bb579640e3ae8d26e73e6782fb` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.469578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b743bb579640e3ae8d26e73e6782fb)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:01.471475 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:01.595955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b743bb579640e3ae8d26e73e6782fb', 'request_id': 'req-1d4f378f-177b-49be-952d-cf03d72b9b51'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:01.604956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b743bb579640e3ae8d26e73e6782fb', 'request_id': 'req-1d4f378f-177b-49be-952d-cf03d72b9b51'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:01.613912 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b743bb579640e3ae8d26e73e6782fb', 'request_id': 'req-1d4f378f-177b-49be-952d-cf03d72b9b51'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:01.740079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] The token cache is being invalidate because project 38b743bb579640e3ae8d26e73e6782fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:01.740286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:01.740944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1d4f378f-177b-49be-952d-cf03d72b9b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:01.742478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1317/2645] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:01 2022] DELETE /identity/v3/projects/38b743bb579640e3ae8d26e73e6782fb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:01.751823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.752531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.752531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.775029 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:01.776524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:01.862028 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1329/2646] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:01.870897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.871295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.871484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.902628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:01.904265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:01.919361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:01.920410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1318/2647] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:01.934761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c403b-ee96-47f8-aba6-a99273590e4e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:01.940439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:01.946112 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.946469 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.947154 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.947719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:01.948085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:01.949949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:01.967038 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1330/2648] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:02:01.972907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:01.983410 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a9135-3d1f-4fdd-afa8-74364443a0a4 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:01.988446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:01.989342 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:01.989630 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:01.989910 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:01.990703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:01.992936 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.108463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1319/2649] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:01 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:02.110187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.124158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7194f-7088-48d0-9147-c16724b207f5 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.198563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:02.235904 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.236527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.236901 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.237178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.238394 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.240678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.359521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1331/2650] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:02 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:02.360832 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.372923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11c55a-510b-4963-bb72-89922003c38e admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.378466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.379897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.380073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.380367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.381213 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.385404 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.398321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1320/2651] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:02.402465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.413579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62830b21-3282-4b54-a69d-54cd4a1357d8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.418694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.419310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.419590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.419864 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.420518 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.422676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a9e41-d7cb-4246-9a27-020ec5577c35 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.436718 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1332/2652] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:02.440492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.452831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc88fd4-56db-4e09-9df7-82fbb855f9f4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.458088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.458855 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.459198 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.459699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] PATH_INFO: `/v3/projects/55923ebc319d4e1c84b566f83942713a/users/193efa41bd0441cbb0bbce20cfa5be30/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.475680 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=55923ebc319d4e1c84b566f83942713a, user_id=193efa41bd0441cbb0bbce20cfa5be30, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.478516 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.552322 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1321/2653] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:02 2022] PUT /identity/v3/projects/55923ebc319d4e1c84b566f83942713a/users/193efa41bd0441cbb0bbce20cfa5be30/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:02:02.563811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.564137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.564371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.596592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] MFA Rules not processed for user `193efa41bd0441cbb0bbce20cfa5be30`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:02.598630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:02.677432 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1333/2654] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:02.685962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.687759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:02.762204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3d6b3-1033-407b-a3bd-7018a279f2ac admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.765984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.766686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.767032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.767368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.777570 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.779189 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adad1e-8603-4c2f-a3f4-84a7c501b69e service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:02.789461 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1322/2655] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:02.809676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:02.875790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e7dee-ef9a-4f40-aba6-b97992158a1b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:02.880937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:02.881573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:02.881862 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:02.882127 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] PATH_INFO: `/v3/users/193efa41bd0441cbb0bbce20cfa5be30` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:02.883272 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] RBAC: Authorizing `identity:delete_user(user_id=193efa41bd0441cbb0bbce20cfa5be30)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:02.885244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:03.076886 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:03.079315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193efa41bd0441cbb0bbce20cfa5be30'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:03.079960 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193efa41bd0441cbb0bbce20cfa5be30'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:03.087668 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193efa41bd0441cbb0bbce20cfa5be30'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:03.179289 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193efa41bd0441cbb0bbce20cfa5be30'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:03.192701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193efa41bd0441cbb0bbce20cfa5be30'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:03.203113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1334/2656] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:02 2022] DELETE /identity/v3/users/193efa41bd0441cbb0bbce20cfa5be30 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:03.204778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:03.205084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:03.205359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:03.219544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:03.225716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:03.227228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:03.311710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1323/2657] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:02:03.324152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:03.326200 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:03.440426 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb83ed-5c70-465a-9f05-feca42ba289a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:03.447652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:03.448803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:03.449095 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:03.449368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:03.464182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:03.466061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ce46e-5111-4519-b4ba-5ea23f6aaa40 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:03.480793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1335/2658] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:03.970619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:03.972433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:03.996639 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7bb4c-ad10-4617-8ed4-bbbe23a05313 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.004157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.005517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.006124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.006808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] PATH_INFO: `/v3/projects/55923ebc319d4e1c84b566f83942713a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.009121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55923ebc319d4e1c84b566f83942713a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.018269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.158951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55923ebc319d4e1c84b566f83942713a', 'request_id': 'req-3077197e-63d1-426e-969c-474247b8fe16'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:04.166320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55923ebc319d4e1c84b566f83942713a', 'request_id': 'req-3077197e-63d1-426e-969c-474247b8fe16'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:04.176861 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55923ebc319d4e1c84b566f83942713a', 'request_id': 'req-3077197e-63d1-426e-969c-474247b8fe16'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:04.267869 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] The token cache is being invalidate because project 55923ebc319d4e1c84b566f83942713a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:04.268161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:04.269121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3077197e-63d1-426e-969c-474247b8fe16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:04.271566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1324/2659] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:03 2022] DELETE /identity/v3/projects/55923ebc319d4e1c84b566f83942713a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:02:04.275951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57199-574b-4e53-9d85-26c236928745 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.276572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57199-574b-4e53-9d85-26c236928745 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.276572 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e57199-574b-4e53-9d85-26c236928745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.308736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-69e57199-574b-4e53-9d85-26c236928745 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:04.310351 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-69e57199-574b-4e53-9d85-26c236928745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:04.390537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1336/2660] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:04.399411 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.399679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.399994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.429003 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:04.430288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:04.440264 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1325/2661] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:04.444357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e57199-574b-4e53-9d85-26c236928745 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.454952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e57199-574b-4e53-9d85-26c236928745 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.462262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.462767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.463061 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.463377 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.463967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:04.464270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.465927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.482524 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1337/2662] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:02:04.493099 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.507846 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec5aa-c274-446c-bd34-a97aa628dd00 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.512969 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.513539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.513819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.514091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.514784 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.516718 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.610139 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1326/2663] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:04 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:04.611211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.623433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a736e8-bf97-4be3-b9fc-499f73516c2f admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.729443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:04.739660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.740279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.740555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.742199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.742199 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.743483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.857646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1338/2664] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:04 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:04.859200 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.871307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f0149-429b-4e27-a7cc-2b356ae1eb06 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.876109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.876830 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.877184 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.877583 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.878902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.880801 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.893949 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1327/2665] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:04.898641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.910516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5281c1b-b3b1-430e-9647-713df24873f7 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.917573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.918392 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.918679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.919005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.919623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.921913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996a3ab-b28f-4501-900f-d852dbe9f3fe admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:04.940590 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1339/2666] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 08:02:04.945460 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:04.957380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbe4a1-fc1e-49b2-b286-b1d7a0d97478 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:04.963447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:04.964105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:04.964386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:04.964678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] PATH_INFO: `/v3/projects/f54cfb83c50b4bedae7c8e861d27cf78/users/896efd8a1daa4227a174d19827aeb5e1/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:04.980757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f54cfb83c50b4bedae7c8e861d27cf78, user_id=896efd8a1daa4227a174d19827aeb5e1, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:04.983343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:05.060693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1328/2667] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:04 2022] PUT /identity/v3/projects/f54cfb83c50b4bedae7c8e861d27cf78/users/896efd8a1daa4227a174d19827aeb5e1/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:05.132797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:05.133065 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:05.133385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:05.166831 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] MFA Rules not processed for user `896efd8a1daa4227a174d19827aeb5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:05.168749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:05.240854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1340/2668] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:05.263607 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:05.269893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:05.318435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:05.366419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a8a2b5-5cd3-4555-874f-794dc48b8559 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:05.379033 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:05.379773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:05.380149 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:05.380476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:05.397370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:05.399078 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a96f9-0f55-400a-b15e-c05e76ffc133 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:05.423507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1329/2669] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:05.451374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:05.568384 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30e7bf-56b7-4a3a-98cb-66c887f2f090 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:05.574892 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:05.575549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:05.575878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:05.576153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] PATH_INFO: `/v3/users/896efd8a1daa4227a174d19827aeb5e1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:05.577412 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] RBAC: Authorizing `identity:delete_user(user_id=896efd8a1daa4227a174d19827aeb5e1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:05.579815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:05.806459 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896efd8a1daa4227a174d19827aeb5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:05.807329 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896efd8a1daa4227a174d19827aeb5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:05.815336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896efd8a1daa4227a174d19827aeb5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:05.821729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:05.947159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896efd8a1daa4227a174d19827aeb5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:05.973847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896efd8a1daa4227a174d19827aeb5e1'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:05.987149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1341/2670] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:05 2022] DELETE /identity/v3/users/896efd8a1daa4227a174d19827aeb5e1 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:05.992072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:05.992628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:05.992844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:06.035455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:06.039554 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:06.178153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1330/2671] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:06.210693 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:06.212323 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:06.419297 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fd59b-ebcf-45c5-9530-5da49fd5beae admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:06.430993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:06.431910 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:06.432336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:06.432780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:06.452496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:06.454380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6578634-5f9e-4ff2-aefc-a1d468f6f488 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:06.474795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1342/2672] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:08.244916 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:08.250644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:08.263040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f83d4a-ac4c-42ed-86c7-89988150a159 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:08.269653 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:08.270678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:08.271174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:08.271611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] PATH_INFO: `/v3/projects/f54cfb83c50b4bedae7c8e861d27cf78` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:08.273723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f54cfb83c50b4bedae7c8e861d27cf78)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:08.277054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:08.421248 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f54cfb83c50b4bedae7c8e861d27cf78', 'request_id': 'req-f0fbe729-74f0-4912-845c-e28cb28c5013'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:08.435617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f54cfb83c50b4bedae7c8e861d27cf78', 'request_id': 'req-f0fbe729-74f0-4912-845c-e28cb28c5013'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:08.447985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f54cfb83c50b4bedae7c8e861d27cf78', 'request_id': 'req-f0fbe729-74f0-4912-845c-e28cb28c5013'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:08.528485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] The token cache is being invalidate because project f54cfb83c50b4bedae7c8e861d27cf78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:08.529133 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:08.529906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-f0fbe729-74f0-4912-845c-e28cb28c5013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:08.532429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1331/2673] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:08 2022] DELETE /identity/v3/projects/f54cfb83c50b4bedae7c8e861d27cf78 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:02:08.538465 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:08.539341 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:08.539785 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:08.584335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:08.587571 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:08.729644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:08.730142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:08.731183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:08.734795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1343/2674] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:08.760171 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:08.763011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:08.785656 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1332/2675] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:08.789975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:08.801832 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f3d45-65a4-4695-9307-ecbeb1c8d837 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:08.812276 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:08.817016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:08.818863 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:08.820548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:08.821687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:08.824628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:08.829940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:08.850782 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1344/2676] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:02:08.857573 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:08.890463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55aa67-bb31-4abf-91ba-27bb011c6d4c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:08.900831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:08.901691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:08.904113 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:08.904471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:08.905296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:08.912547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.067216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1333/2677] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:08 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:02:09.073143 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.089159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2abce-66df-4d88-bd84-43fe2e1c3b15 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.174646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.174914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.175161 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.175438 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.176106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.178349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.287312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:09.296583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1345/2678] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:09 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:09.297833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.309378 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c832b06-4339-4ff8-b1e3-c18dca7f4860 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.314291 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.314902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.315182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.315454 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.316176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.317966 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.336706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1334/2679] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:09.340300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.351164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd72fc9-0cae-4b91-a09c-2e2ea8821b73 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.356694 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.357300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.357579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.357879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.358531 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.360125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59384f5f-43ec-4516-8be4-78dfa5695b32 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.371626 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1346/2680] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:09.375640 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.387524 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b991b-85de-4982-8df7-e7c910c99c81 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.391983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.392430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.392745 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.393027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] PATH_INFO: `/v3/projects/d6dc895a3723479796c3bf0683866be1/users/a9dc297b8f9f4691a945a7521ce43515/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.409059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:09.413093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dc895a3723479796c3bf0683866be1, user_id=a9dc297b8f9f4691a945a7521ce43515, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.415828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.506020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1335/2681] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:09 2022] PUT /identity/v3/projects/d6dc895a3723479796c3bf0683866be1/users/a9dc297b8f9f4691a945a7521ce43515/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:09.516385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.516689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.516965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.545192 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] MFA Rules not processed for user `a9dc297b8f9f4691a945a7521ce43515`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:09.546338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:09.613287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1347/2682] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:09.618287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.714383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67932143-bffa-4473-88a0-c571d1528683 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.720739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.720739 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.721245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.721245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.721776 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.723715 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:09.832365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1336/2683] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:09 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:09.836840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:09.848641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e5189-f29d-4a50-8068-70d1b8847a49 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:09.908027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:09.908667 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:09.908953 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:09.909244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:09.909920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:09.911914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.005908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1348/2684] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:09 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:10.007153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.017756 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bd247-a3f0-41f5-a731-178539060e0c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.023069 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.030557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.030557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.030557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.030557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.030557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.038354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1337/2685] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:10.042451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.052635 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9a598-971e-48d5-8d95-59ec279c6aa8 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.057846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.058448 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.058729 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.059005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.059646 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.061287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d6d0f-eb74-41f9-8bbd-5c597faa666a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.071724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1349/2686] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:10.075893 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.088591 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2bb62-ed32-4210-aeda-0cde53453373 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.094233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.094871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.095154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.095428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] PATH_INFO: `/v3/projects/ec886905374f4a3d81e29b7cd62ba1b9/users/5ce99bf8cea04127a91621d1eb7725a3/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.112637 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec886905374f4a3d81e29b7cd62ba1b9, user_id=5ce99bf8cea04127a91621d1eb7725a3, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.116283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.214538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1338/2687] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:10 2022] PUT /identity/v3/projects/ec886905374f4a3d81e29b7cd62ba1b9/users/5ce99bf8cea04127a91621d1eb7725a3/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:10.225793 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.226115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.226393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.255929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] MFA Rules not processed for user `5ce99bf8cea04127a91621d1eb7725a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:10.257219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:10.304495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:10.326250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1350/2688] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:10.330841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.397471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcb5d-7cbb-42de-82b4-72e2c1c17b0d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.410742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.410742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.410742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.410742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.411703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.411703 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.523445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1339/2689] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:10 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:10.527774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.539356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfada58-b98b-483d-aa41-e0cb8d16171a None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.594084 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.594736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.594865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.595137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.595761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.598393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.665415 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1351/2690] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:10 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:10.669563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.680029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e73d5-0d52-4d83-a945-b48667ac7be8 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.684728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.685389 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.685673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.685945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.686615 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.688206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.699076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1340/2691] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:10.703256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.714439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2948-c6b0-421a-894b-2cd9d5cdcc04 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.720241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.720947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.721240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.721504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] PATH_INFO: `/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf/users/30b7bacbd8e44eeab2311aee9ce7c942/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.738850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f4c17307e14b9688b37bc5bd6e5cbf, user_id=30b7bacbd8e44eeab2311aee9ce7c942, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.741592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.806630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1352/2692] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:10 2022] PUT /identity/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf/users/30b7bacbd8e44eeab2311aee9ce7c942/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:02:10.811204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.822068 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1f844-8141-497b-961f-d3016feb44d3 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.832735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:10.884432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.885176 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.886268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.886984 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.887753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.889728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e8b11-a377-487a-b675-3a30b5a429ca admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.905702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1341/2693] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:10.907261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:10.920795 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ccd28-5d33-4270-9c59-a36851e83a50 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:10.925903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:10.926614 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.926894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.927165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] PATH_INFO: `/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf/users/30b7bacbd8e44eeab2311aee9ce7c942/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.929157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f4c17307e14b9688b37bc5bd6e5cbf, user_id=30b7bacbd8e44eeab2311aee9ce7c942, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:10.931870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:10.943780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1353/2694] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:10 2022] PUT /identity/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf/users/30b7bacbd8e44eeab2311aee9ce7c942/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:10.954414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:10.954727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:10.955005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:10.985343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] MFA Rules not processed for user `30b7bacbd8e44eeab2311aee9ce7c942`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:10.986596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:11.068298 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1342/2695] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:11.079900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:11.082266 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:11.142558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0a287-49b0-4eeb-ba93-e0b6b2ca822e admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:11.147869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:11.148495 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:11.148809 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:11.149088 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:11.222439 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:11.239234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:11.239234 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:11.248850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1354/2696] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:13.337214 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:13.347875 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3009fea-c535-45ca-84bf-173e5c1d8963 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:13.351551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:13.352152 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:13.352432 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:13.352746 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:13.412012 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:13.413470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43f39b-43bd-41fe-8887-509f41163e65 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:13.422588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1343/2697] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:15.037912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:15.046763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0014e3b-bb24-49f8-b16a-7f4d4fe51f74 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:15.051840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:15.051840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:15.051840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:15.052336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:15.062786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:15.064218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:15.075533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1355/2698] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:17.151702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b229-3e37-48bf-a4ee-662ff2fa1931 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:17.152174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b229-3e37-48bf-a4ee-662ff2fa1931 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:17.152397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740b229-3e37-48bf-a4ee-662ff2fa1931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:17.172762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-a740b229-3e37-48bf-a4ee-662ff2fa1931 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:17.174081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-a740b229-3e37-48bf-a4ee-662ff2fa1931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:17.209897 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=100 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:17.245963 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1344/2699] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 08:02:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:17.261433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:17.263000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:17.335395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d71f-f7e2-4266-b648-fed9ba6044ec service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:17.340711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:17.341309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:17.341653 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:17.341988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:17.351141 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:17.353085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:17.361533 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1356/2700] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 08:02:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:02:18.898676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.899153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.899383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.919245 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] MFA Rules not processed for user `0c877a807efb45a8b6475e0c8db99e98`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:18.921670 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:18.928355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1345/2701] 10.222.0.72 () {58 vars in 967 bytes} [Wed Aug 24 08:02:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:18.941867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:18.962194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c282fc1-ceb6-4db5-9a98-c31528ab004e service swift] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:18.962652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e977eb2e34a945b4adeffbc92d154248', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:18.965652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:18.965940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:18.966219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:18.975861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:18.977630 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4224-1f69-4aaf-acbf-691e62771151 service swift] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:18.987115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1357/2702] 10.222.0.72 () {60 vars in 1384 bytes} [Wed Aug 24 08:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 08:02:20.083727 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:20.149677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06530449-e046-402f-9aee-1c0ffa1b6fb0 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:20.154584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:20.155223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:20.155502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:20.155779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] PATH_INFO: `/v3/users/a9dc297b8f9f4691a945a7521ce43515` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:20.157246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9dc297b8f9f4691a945a7521ce43515)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:20.159182 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:20.410006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9dc297b8f9f4691a945a7521ce43515'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:20.417186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9dc297b8f9f4691a945a7521ce43515'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:20.417911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9dc297b8f9f4691a945a7521ce43515'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:20.418515 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9dc297b8f9f4691a945a7521ce43515'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:20.553937 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9dc297b8f9f4691a945a7521ce43515'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:20.575587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1346/2703] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:20 2022] DELETE /identity/v3/users/a9dc297b8f9f4691a945a7521ce43515 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:20.580183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:20.580755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:20.580755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:20.601244 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:20.602612 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:20.665017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1358/2704] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:20.674876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:20.677164 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:20.735604 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915b640-a9bb-498c-88ac-9d30f94fc56c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:20.740018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:20.740628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:20.740940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:20.741238 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:20.747556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:20.751520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:20.752989 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:20.762955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1347/2705] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:21.266847 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.268380 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:21.284324 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd080ab-41fc-4fcd-b28f-579d3e0c34f7 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.293824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.294719 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.295002 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.295309 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] PATH_INFO: `/v3/projects/d6dc895a3723479796c3bf0683866be1` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.298924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dc895a3723479796c3bf0683866be1)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.301041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.460889 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dc895a3723479796c3bf0683866be1', 'request_id': 'req-63ac0252-e5a4-4bee-8a63-e85a459aedff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.462555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:21.539195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dc895a3723479796c3bf0683866be1', 'request_id': 'req-63ac0252-e5a4-4bee-8a63-e85a459aedff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.551645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dc895a3723479796c3bf0683866be1', 'request_id': 'req-63ac0252-e5a4-4bee-8a63-e85a459aedff'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.563848 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] The token cache is being invalidate because project d6dc895a3723479796c3bf0683866be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:21.564139 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.564898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.567073 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1359/2706] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:21 2022] DELETE /identity/v3/projects/d6dc895a3723479796c3bf0683866be1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:21.571057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:21.582605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f3bf7-0e1b-4b7c-99af-55c8caba904e service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:21.654316 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:21.655306 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:21.655724 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:21.656258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] PATH_INFO: `/v3/users/5ce99bf8cea04127a91621d1eb7725a3` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:21.657481 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce99bf8cea04127a91621d1eb7725a3)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:21.659584 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:21.976118 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce99bf8cea04127a91621d1eb7725a3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.984366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce99bf8cea04127a91621d1eb7725a3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.985699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce99bf8cea04127a91621d1eb7725a3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:21.986343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce99bf8cea04127a91621d1eb7725a3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.079544 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce99bf8cea04127a91621d1eb7725a3'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.101942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1348/2707] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:21 2022] DELETE /identity/v3/users/5ce99bf8cea04127a91621d1eb7725a3 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:22.581104 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:22.582876 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:22.597478 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac0252-e5a4-4bee-8a63-e85a459aedff admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:22.674375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:22.675153 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:22.675506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:22.675861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] PATH_INFO: `/v3/projects/ec886905374f4a3d81e29b7cd62ba1b9` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:22.677947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec886905374f4a3d81e29b7cd62ba1b9)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:22.679992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:22.853103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec886905374f4a3d81e29b7cd62ba1b9', 'request_id': 'req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.962451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec886905374f4a3d81e29b7cd62ba1b9', 'request_id': 'req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.969900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec886905374f4a3d81e29b7cd62ba1b9', 'request_id': 'req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.983763 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] The token cache is being invalidate because project ec886905374f4a3d81e29b7cd62ba1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:22.984133 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.984854 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:22.987030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1360/2708] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:22 2022] DELETE /identity/v3/projects/ec886905374f4a3d81e29b7cd62ba1b9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:22.991835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:22.993380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:23.006877 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb71b10-2027-432a-bfb9-5e32929f164c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:23.064831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:23.065463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:23.065744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:23.066016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] PATH_INFO: `/v3/users/30b7bacbd8e44eeab2311aee9ce7c942` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:23.067167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b7bacbd8e44eeab2311aee9ce7c942)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:23.069151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:23.082589 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:23.321240 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b7bacbd8e44eeab2311aee9ce7c942'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:23.345658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b7bacbd8e44eeab2311aee9ce7c942'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:23.346437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b7bacbd8e44eeab2311aee9ce7c942'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:23.347121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b7bacbd8e44eeab2311aee9ce7c942'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:23.446121 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-bd1da516-5ede-48c1-8a05-0747cdd45daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b7bacbd8e44eeab2311aee9ce7c942'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:23.476734 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1349/2709] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:22 2022] DELETE /identity/v3/users/30b7bacbd8e44eeab2311aee9ce7c942 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:24.061751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:24.062815 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.081191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4030ec0-4a9d-47ed-b7b6-f4ab8a24ea13 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:24.162720 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:24.164906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.164906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.164906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] PATH_INFO: `/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.166326 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f4c17307e14b9688b37bc5bd6e5cbf)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:24.168367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:24.179728 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:24.324112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5731-b4ea-4b58-ad5b-d3c222e1b1a5 None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.325526 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5731-b4ea-4b58-ad5b-d3c222e1b1a5 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.326227 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a5731-b4ea-4b58-ad5b-d3c222e1b1a5 None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.329191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1350/2710] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 08:02:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:02:24.335686 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcecea-f1fb-4aa0-97b6-67414c1f99a4 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.336036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f4c17307e14b9688b37bc5bd6e5cbf', 'request_id': 'req-c9a76294-8362-4508-a4d2-b4d38be536dd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:24.336795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcecea-f1fb-4aa0-97b6-67414c1f99a4 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.337539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcecea-f1fb-4aa0-97b6-67414c1f99a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.365835 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-76bcecea-f1fb-4aa0-97b6-67414c1f99a4 None None] MFA Rules not processed for user `c8651a702d1d4681be3ed1ac16ce9b39`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:24.367073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-76bcecea-f1fb-4aa0-97b6-67414c1f99a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.424581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f4c17307e14b9688b37bc5bd6e5cbf', 'request_id': 'req-c9a76294-8362-4508-a4d2-b4d38be536dd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:24.437915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f4c17307e14b9688b37bc5bd6e5cbf', 'request_id': 'req-c9a76294-8362-4508-a4d2-b4d38be536dd'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:24.453477 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] The token cache is being invalidate because project f4f4c17307e14b9688b37bc5bd6e5cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:24.453850 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:24.454688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:24.456496 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1361/2711] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:24 2022] DELETE /identity/v3/projects/f4f4c17307e14b9688b37bc5bd6e5cbf => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:24.462561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1351/2712] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 08:02:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:24.480684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:24.484321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.526802 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a76294-8362-4508-a4d2-b4d38be536dd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:24.531753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:24.536770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.537559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.538032 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.541883 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.557135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:24.558804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b1a68-3abf-4c28-a129-cd705d2eb4b9 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:24.572410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1362/2713] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:24.713543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cee6a-c610-4cf6-af62-6669c4e1e47b None None] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.714438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cee6a-c610-4cf6-af62-6669c4e1e47b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.716390 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cee6a-c610-4cf6-af62-6669c4e1e47b None None] PATH_INFO: `/` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.719296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1352/2714] 10.222.0.72 () {54 vars in 921 bytes} [Wed Aug 24 08:02:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 08:02:24.731006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.731321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.731592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.752487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] MFA Rules not processed for user `d882cbdd492344c7a3638c65ed08b32d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:24.753810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:24.827285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1363/2715] 10.222.0.72 () {58 vars in 1018 bytes} [Wed Aug 24 08:02:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:24.837990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cee6a-c610-4cf6-af62-6669c4e1e47b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:24.849289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cee6a-c610-4cf6-af62-6669c4e1e47b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:24.853764 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:24.854520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:24.854909 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:24.855288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:24.868100 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:24.870782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:24.883295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1353/2716] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:28.810705 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:28.812156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:28.876838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c33f2-418a-474d-b649-a3a95705acae None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:28.881804 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:28.882423 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:28.882695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:28.882959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] PATH_INFO: `/v3/users/b3fa420e4c0049b9ae5cee7717422cda` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:28.884148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fa420e4c0049b9ae5cee7717422cda)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:28.886096 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:28.907328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:29.096030 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fa420e4c0049b9ae5cee7717422cda'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.096712 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fa420e4c0049b9ae5cee7717422cda'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.104074 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fa420e4c0049b9ae5cee7717422cda'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.166615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fa420e4c0049b9ae5cee7717422cda'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.179787 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fa420e4c0049b9ae5cee7717422cda'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.185895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1364/2717] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:28 2022] DELETE /identity/v3/users/b3fa420e4c0049b9ae5cee7717422cda => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:29.494488 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:29.507906 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017abc5-359d-432b-aaad-44af6cb34d84 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:29.567151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844425-4f95-4758-82be-392711188864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:29.567785 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844425-4f95-4758-82be-392711188864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:29.568066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844425-4f95-4758-82be-392711188864 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:29.568350 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da844425-4f95-4758-82be-392711188864 admin admin] PATH_INFO: `/v3/projects/4b41efe770974b3aa9a34e21dc2ea5d2` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:29.569597 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da844425-4f95-4758-82be-392711188864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b41efe770974b3aa9a34e21dc2ea5d2)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:29.571596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da844425-4f95-4758-82be-392711188864 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:29.669345 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:29.681659 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b41efe770974b3aa9a34e21dc2ea5d2', 'request_id': 'req-da844425-4f95-4758-82be-392711188864'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.687014 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b41efe770974b3aa9a34e21dc2ea5d2', 'request_id': 'req-da844425-4f95-4758-82be-392711188864'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.693619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b41efe770974b3aa9a34e21dc2ea5d2', 'request_id': 'req-da844425-4f95-4758-82be-392711188864'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.761057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] The token cache is being invalidate because project 4b41efe770974b3aa9a34e21dc2ea5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:29.761347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.761971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-da844425-4f95-4758-82be-392711188864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:29.763509 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1354/2718] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:29 2022] DELETE /identity/v3/projects/4b41efe770974b3aa9a34e21dc2ea5d2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:29.767522 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:29.768955 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:29.782755 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44732488-8fec-4156-8f5c-381861843c6b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:29.839746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:29.840435 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:29.840716 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:29.841010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] PATH_INFO: `/v3/users/002d566075f54467b5db9cb6eca80fd7` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:29.842138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002d566075f54467b5db9cb6eca80fd7)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:29.844053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:30.039219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002d566075f54467b5db9cb6eca80fd7'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.039881 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002d566075f54467b5db9cb6eca80fd7'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.046992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002d566075f54467b5db9cb6eca80fd7'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.109405 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002d566075f54467b5db9cb6eca80fd7'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.121746 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-3fcd4198-2385-4ef5-bef4-4089a15ed935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002d566075f54467b5db9cb6eca80fd7'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.127642 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1365/2719] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:02:29 2022] DELETE /identity/v3/users/002d566075f54467b5db9cb6eca80fd7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:30.391230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844425-4f95-4758-82be-392711188864 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:30.392699 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-da844425-4f95-4758-82be-392711188864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:30.409961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da844425-4f95-4758-82be-392711188864 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:30.473428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:30.473895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:30.474172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:30.474439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] PATH_INFO: `/v3/projects/2b9ec53274cc4c8c8e54c3811b712626` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:30.475693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b9ec53274cc4c8c8e54c3811b712626)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:30.477791 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:30.581623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b9ec53274cc4c8c8e54c3811b712626', 'request_id': 'req-df051e17-07cc-4bd8-a545-fc2a08ccfc43'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.588397 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b9ec53274cc4c8c8e54c3811b712626', 'request_id': 'req-df051e17-07cc-4bd8-a545-fc2a08ccfc43'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.595045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b9ec53274cc4c8c8e54c3811b712626', 'request_id': 'req-df051e17-07cc-4bd8-a545-fc2a08ccfc43'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.692859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] The token cache is being invalidate because project 2b9ec53274cc4c8c8e54c3811b712626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:02:30.693054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.693689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-df051e17-07cc-4bd8-a545-fc2a08ccfc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:02:30.695202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1355/2720] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:02:30 2022] DELETE /identity/v3/projects/2b9ec53274cc4c8c8e54c3811b712626 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:30.700221 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:30.700540 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:30.701050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:30.720764 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:30.721981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:30.731615 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:30.793687 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1366/2721] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:30.802336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:30.802617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:30.802885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:30.823321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:30.824514 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:30.832475 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1356/2722] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:30.836553 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:30.847071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2888c13-c8b4-49dc-9dff-4d97f25ec760 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:30.851723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:30.852319 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:30.852600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:30.852902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:30.853551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:02:30.853858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:30.855451 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:30.867357 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1367/2723] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:02:30.871794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:30.881664 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab0037-f39c-4fb1-8936-32f439937ce8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:30.886354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:30.886947 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:30.887226 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:30.887495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:30.888153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:30.890058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.003622 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1357/2724] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:30 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:31.007717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:31.018017 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a6a64-0868-49df-9bb4-3fcdec1e933d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:31.087011 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:31.087645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.087947 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.088219 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.088885 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:31.090836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.205239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1368/2725] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:31 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:02:31.209586 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:31.220074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17686c-70a2-4e6a-a3e2-ff97a5568280 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:31.225201 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:31.225795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.226073 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.226347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.227017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:31.228625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.239098 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1358/2726] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:31.243240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:31.253135 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31bba3-47e2-4518-b441-22d47a761dd2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:31.257836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:31.258371 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.258652 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.258927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.259568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:31.261195 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7b402-d180-4469-87bb-9fbb528fddf3 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.272528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1369/2727] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:31.276625 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:31.287418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618cfd9-5ba7-4952-aa3f-d510bcd491eb admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:31.292000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:31.292643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.292957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.293234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] PATH_INFO: `/v3/projects/67da3e4c41e64947bb3e531c5b7550b6/users/7a279965532e4020876954116f53d3b9/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.296232 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:31.308216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67da3e4c41e64947bb3e531c5b7550b6, user_id=7a279965532e4020876954116f53d3b9, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:31.310874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.393669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1359/2728] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:31 2022] PUT /identity/v3/projects/67da3e4c41e64947bb3e531c5b7550b6/users/7a279965532e4020876954116f53d3b9/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:31.400961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325a155-8dea-4480-8378-75f0df81a55f None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.401420 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325a155-8dea-4480-8378-75f0df81a55f None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.401825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325a155-8dea-4480-8378-75f0df81a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.429837 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5325a155-8dea-4480-8378-75f0df81a55f None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:31.431342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5325a155-8dea-4480-8378-75f0df81a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:31.495767 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1370/2729] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:31.505466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:31.507212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:31.565881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bdcf3-434d-4b06-9d70-515c42922179 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:31.569471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:31.570070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:31.570347 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:31.570646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:31.579508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:31.580956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:31.590705 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1360/2730] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:37.037686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.038197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.038443 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.075226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] MFA Rules not processed for user `7a279965532e4020876954116f53d3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:37.075226 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:37.080335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:37.160707 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1371/2731] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:37.166806 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.180363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aad8-4ca6-4681-99d2-19452351d56a service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.184428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.185428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.185547 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.185852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.186590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.189814 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.284813 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1361/2732] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:02:37 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:37.288046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.299770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c411c-ac49-46e8-a1f9-581596e19d03 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.374196 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.375462 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.375811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.376157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.377304 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.379997 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.491741 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1372/2733] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:02:37 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:02:37.493077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.504977 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7ce2c-0a0d-4474-9f47-b7e6041a8b2f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.509356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.509979 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.510252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.510523 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.511170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.512810 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.524802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1362/2734] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:37.528924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.540151 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d64357-995e-46a0-b3f6-232f25b2d4e5 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.544873 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.545556 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.545834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.546136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] PATH_INFO: `/v3/projects/cbf42a2d4f154881afaeb177584af04e/users/5a00deec80c542c6a21b04f1e3132105/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.562494 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf42a2d4f154881afaeb177584af04e, user_id=5a00deec80c542c6a21b04f1e3132105, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.565210 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.637870 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1373/2735] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:37 2022] PUT /identity/v3/projects/cbf42a2d4f154881afaeb177584af04e/users/5a00deec80c542c6a21b04f1e3132105/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:37.648088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.654934 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30a236-bc1d-442f-b7b9-1413a598d8e0 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.726335 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.727210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.727210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.727491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.728142 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.729780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dac244-be1b-4dcb-a270-38ac14ac5321 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.741011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1363/2736] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:02:37.745187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:37.758285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ba6d2-d187-4351-8a67-8ffa89004a02 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:37.763551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:37.764283 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:37.764869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:37.765145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] PATH_INFO: `/v3/projects/cbf42a2d4f154881afaeb177584af04e/users/5a00deec80c542c6a21b04f1e3132105/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:37.771906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf42a2d4f154881afaeb177584af04e, user_id=5a00deec80c542c6a21b04f1e3132105, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:37.772681 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:37.785070 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1374/2737] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:02:37 2022] PUT /identity/v3/projects/cbf42a2d4f154881afaeb177584af04e/users/5a00deec80c542c6a21b04f1e3132105/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:02:43.912196 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:43.912774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:43.912774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:43.944932 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] MFA Rules not processed for user `5a00deec80c542c6a21b04f1e3132105`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:02:43.946170 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:43.995130 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:44.026831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1364/2738] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:02:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:02:44.040996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:02:44.042911 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:02:44.115386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:44.122583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3e94e-b96e-4479-b5fc-1533691ea891 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:02:44.128937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:02:44.129623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:02:44.129906 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:02:44.130182 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:02:44.192012 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:02:44.193671 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198bfb-c39b-4b93-97f2-2d98ef585088 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:02:44.208170 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1375/2739] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:02:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:02:54.081283 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=11 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:02:54.201437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=27 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:00.207331 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:00.208775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:00.266370 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be472b37-15a0-4d85-a924-fdb74012225c None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:00.270001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:00.270612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:00.270891 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:00.271183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:00.273540 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:00.338015 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:00.339464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:00.349295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1365/2740] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:03.623871 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:03.629560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:03.629560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:03.645261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] MFA Rules not processed for user `7a279965532e4020876954116f53d3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:03.646722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:03.654788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1376/2741] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:03:04.029007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:04.038258 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545bcd-13e5-4790-bdc8-00a51669ba14 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:04.042136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:04.042742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:04.043037 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:04.043300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:04.045083 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:04.055333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:04.056902 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:04.066247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1366/2742] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:04.142269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:04.144041 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:04.161140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaf901-fb7e-4027-af42-a02c0122fcd1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:04.165915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:04.166560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:04.166838 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:04.167137 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] PATH_INFO: `/v3/users/d318793a5fee43bdb1a043b30396884b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:04.168308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d318793a5fee43bdb1a043b30396884b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:04.170264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:04.365308 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd318793a5fee43bdb1a043b30396884b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:04.366413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd318793a5fee43bdb1a043b30396884b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:04.373514 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd318793a5fee43bdb1a043b30396884b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:04.430334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd318793a5fee43bdb1a043b30396884b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:04.446346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4edb19eb-cd15-4c73-bb0e-b95a1406ddc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd318793a5fee43bdb1a043b30396884b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:04.453988 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1377/2743] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:03:04 2022] DELETE /identity/v3/users/d318793a5fee43bdb1a043b30396884b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:03:04.751470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:04.770441 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598432b-1137-4b48-86cc-16c060f81b29 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:04.967811 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:04.968455 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:04.968899 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:04.969361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] PATH_INFO: `/v3/projects/27c8e372420a4076b855d46a4f8ff521` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:04.970971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27c8e372420a4076b855d46a4f8ff521)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:04.972920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.080881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:05.088177 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27c8e372420a4076b855d46a4f8ff521', 'request_id': 'req-b09edc0c-e445-408d-9889-fc585594800e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:05.093678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27c8e372420a4076b855d46a4f8ff521', 'request_id': 'req-b09edc0c-e445-408d-9889-fc585594800e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:05.099778 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27c8e372420a4076b855d46a4f8ff521', 'request_id': 'req-b09edc0c-e445-408d-9889-fc585594800e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:05.200212 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] The token cache is being invalidate because project 27c8e372420a4076b855d46a4f8ff521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:03:05.200506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:05.201167 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b09edc0c-e445-408d-9889-fc585594800e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:03:05.202674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1367/2744] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:03:04 2022] DELETE /identity/v3/projects/27c8e372420a4076b855d46a4f8ff521 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:03:05.207789 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.208250 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.208367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.230131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:05.231331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:05.294539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1378/2745] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:03:05.302743 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.303070 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.303394 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.324333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:05.325693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:05.334187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1368/2746] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:03:05.339168 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.351440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7c1c4-de40-4ad2-bf0a-a3b6ba748643 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.356211 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.360868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.372985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1379/2747] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 08:03:05.377758 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.388246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606e688-fed3-4a2f-8926-9672042b07d7 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.397845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.397845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.397845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.397845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.398393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.398393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.477624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1369/2748] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:03:05 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:03:05.481636 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.492573 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0527cd-5a1e-47d2-9d16-49b644e250f4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.545934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:05.555356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.556092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.556372 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.556641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.557303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.559558 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.692236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1380/2749] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:03:05 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:03:05.696959 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.707406 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ede154-01fd-446e-b61c-371f44fc4bb4 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.712827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.713484 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.713765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.714064 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.714710 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.716296 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.727253 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1370/2750] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:03:05.731015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.741296 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c574b-4486-4ae7-aaa1-8d831cfcffbd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.746103 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.746701 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.746980 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.747300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.747949 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.749628 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007df04-24f0-452c-8029-209d40ad455c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.764393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1381/2751] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:03:05.765677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.776965 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9848d-8136-403d-bf10-e8f6e612982c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:05.781823 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:05.782593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.782871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.787443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] PATH_INFO: `/v3/projects/ef79c9df5a2541afaeff1a82b3e4a496/users/a590d584ed654ca399875a5e04a57a24/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.797946 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79c9df5a2541afaeff1a82b3e4a496, user_id=a590d584ed654ca399875a5e04a57a24, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:05.800692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:05.892328 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1371/2752] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:03:05 2022] PUT /identity/v3/projects/ef79c9df5a2541afaeff1a82b3e4a496/users/a590d584ed654ca399875a5e04a57a24/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:03:05.897519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9982b-9dd4-4655-aa7e-e6ecbb892556 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:05.897813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9982b-9dd4-4655-aa7e-e6ecbb892556 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:05.898080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9982b-9dd4-4655-aa7e-e6ecbb892556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:05.916972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-71c9982b-9dd4-4655-aa7e-e6ecbb892556 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:05.918187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-71c9982b-9dd4-4655-aa7e-e6ecbb892556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:05.978504 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1382/2753] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:03:05.988321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:05.990053 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:06.057428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d755650-7502-4401-be3d-45da8fe46078 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:06.061146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:06.061944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:06.062217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:06.062486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:06.072156 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:06.073623 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:06.083146 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1372/2754] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:03:13.213915 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366bf0-dfbe-4aac-b6f1-e25cc579665c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:13.215062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366bf0-dfbe-4aac-b6f1-e25cc579665c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:13.215344 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366bf0-dfbe-4aac-b6f1-e25cc579665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:13.250282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-31366bf0-dfbe-4aac-b6f1-e25cc579665c None None] MFA Rules not processed for user `a590d584ed654ca399875a5e04a57a24`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:13.252100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-31366bf0-dfbe-4aac-b6f1-e25cc579665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:13.357364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1383/2755] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:03:13.365134 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:13.367279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:13.445668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:13.450365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8ba6a-e2b3-422e-ab51-574cb73785ba service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:13.455896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:13.463036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:13.463036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:13.463036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:13.465224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:13.466792 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:13.475876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1373/2756] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:15.923568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:15.925385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:15.925956 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:15.956093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] MFA Rules not processed for user `a590d584ed654ca399875a5e04a57a24`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:03:15.958051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:15.967991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1384/2757] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:03:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:03:16.305203 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:16.315307 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c3a2f-d132-4bab-b307-153e2659bd47 service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:16.319022 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:16.319692 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:16.319970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:16.320244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:16.322188 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:16.332621 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:16.334272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:16.343929 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1374/2758] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:24.341090 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:24.350836 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c067c-e8b0-46df-9599-6aab8c4652e2 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:24.354396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:24.354983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:24.355287 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:24.355555 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:24.357257 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:24.408374 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:24.420623 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:24.422162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:24.432743 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1385/2759] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:26.339794 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=10 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:34.462867 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:03:40.764265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:03:40.765747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:40.905437 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711e3f1-42cd-4887-b37e-b6c2a2979d0e service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:03:40.910372 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:03:40.910968 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:03:40.911246 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:03:40.911517 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:03:40.913254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:03:40.925476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:03:40.927110 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:03:40.936870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1375/2760] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:03:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:03:50.932733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=63 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:00.651631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:00.684092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae14f89-f500-4f48-adf5-609a832fc22e service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:00.688488 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:00.689134 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:00.689418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:00.689695 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:00.693187 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:00.729884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:00.732851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:00.751262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1386/2761] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 08:04:07.943713 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:07.945737 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:07.964684 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df009275-f588-4bfc-8457-beceef41cd6b service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:07.970355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:07.970995 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:07.971272 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:07.971541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] PATH_INFO: `/v3/users/0ae325c8314c47a384a83509e3c281a0` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:07.972673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae325c8314c47a384a83509e3c281a0)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:07.974611 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:08.241610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae325c8314c47a384a83509e3c281a0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.249233 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae325c8314c47a384a83509e3c281a0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.249944 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae325c8314c47a384a83509e3c281a0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.250551 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae325c8314c47a384a83509e3c281a0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.336685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae325c8314c47a384a83509e3c281a0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.354063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1376/2762] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:04:07 2022] DELETE /identity/v3/users/0ae325c8314c47a384a83509e3c281a0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:04:08.604282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:08.631662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396b8fe-6b9e-43da-8605-353a3393e71f service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:08.689163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:08.689788 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:08.690060 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:08.690327 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] PATH_INFO: `/v3/projects/576781ceab42470e9fd28a32ccb29e7f` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:08.691428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=576781ceab42470e9fd28a32ccb29e7f)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:08.693385 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:08.806188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '576781ceab42470e9fd28a32ccb29e7f', 'request_id': 'req-73e7ec2f-c354-4995-a750-e60207962237'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.902355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '576781ceab42470e9fd28a32ccb29e7f', 'request_id': 'req-73e7ec2f-c354-4995-a750-e60207962237'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.909457 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '576781ceab42470e9fd28a32ccb29e7f', 'request_id': 'req-73e7ec2f-c354-4995-a750-e60207962237'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.920631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] The token cache is being invalidate because project 576781ceab42470e9fd28a32ccb29e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:04:08.920940 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.921568 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:08.923049 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1387/2763] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:04:08 2022] DELETE /identity/v3/projects/576781ceab42470e9fd28a32ccb29e7f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:04:08.927065 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:08.928473 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:08.942348 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ef87-7d3f-4536-b413-7cb2f3def3f1 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:09.000215 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:09.001019 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:09.001183 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:09.001456 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] PATH_INFO: `/v3/users/de771603647a4445bac640f6cec4c3a4` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:09.002632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de771603647a4445bac640f6cec4c3a4)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:09.004595 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:09.287802 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de771603647a4445bac640f6cec4c3a4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.295673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de771603647a4445bac640f6cec4c3a4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.296414 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de771603647a4445bac640f6cec4c3a4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.297163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de771603647a4445bac640f6cec4c3a4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.364050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de771603647a4445bac640f6cec4c3a4'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.376269 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:09.384208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1377/2764] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:04:08 2022] DELETE /identity/v3/users/de771603647a4445bac640f6cec4c3a4 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:04:09.652162 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:04:09.653833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:04:09.667338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ec2f-c354-4995-a750-e60207962237 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:04:09.695908 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:09.726649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:04:09.727352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:04:09.727631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:04:09.727903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] PATH_INFO: `/v3/projects/daa7ded529644ce7866f85298f192831` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:04:09.729106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa7ded529644ce7866f85298f192831)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:04:09.731009 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:04:09.869506 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa7ded529644ce7866f85298f192831', 'request_id': 'req-4908679f-6b68-444f-8f5b-2ce02b392b39'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.942228 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa7ded529644ce7866f85298f192831', 'request_id': 'req-4908679f-6b68-444f-8f5b-2ce02b392b39'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.949152 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa7ded529644ce7866f85298f192831', 'request_id': 'req-4908679f-6b68-444f-8f5b-2ce02b392b39'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.959638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] The token cache is being invalidate because project daa7ded529644ce7866f85298f192831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:04:09.959927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.960569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-4908679f-6b68-444f-8f5b-2ce02b392b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:04:09.962079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1388/2765] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:04:09 2022] DELETE /identity/v3/projects/daa7ded529644ce7866f85298f192831 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:04:19.502598 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:04:19.961826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:05:26.747447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:05:26.748999 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:26.831562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3722d-8e16-4c44-a48b-e2be51447c4b admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:05:26.888942 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:05:26.889641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:05:26.889990 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:05:26.890375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:05:26.892261 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:27.010219 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:05:27.011693 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:05:27.043295 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1378/2766] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:05:28.160339 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a7a6-9640-46e8-864f-12085822eeac None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:05:28.164379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a7a6-9640-46e8-864f-12085822eeac None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:05:28.164379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a7a6-9640-46e8-864f-12085822eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:05:28.180058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0264a7a6-9640-46e8-864f-12085822eeac None None] MFA Rules not processed for user `7a279965532e4020876954116f53d3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:05:28.182102 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0264a7a6-9640-46e8-864f-12085822eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:28.238648 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1389/2767] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:05:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:05:29.316941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:05:29.318405 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:29.329066 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bb2ae2-4336-403b-856b-41d770978a14 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:05:29.333187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:05:29.333782 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:05:29.334571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:05:29.334845 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:05:29.336712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:29.351613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:05:29.352881 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:05:29.365023 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1379/2768] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:05:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:05:38.231925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=25 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:05:39.359048 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:05:57.411178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:05:57.411724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:05:57.411724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:05:57.431059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] MFA Rules not processed for user `a590d584ed654ca399875a5e04a57a24`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:05:57.432264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:05:57.486999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1390/2769] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:05:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:06:01.178080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:06:01.187581 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2eec9-ea8d-42c0-bc8f-4fb636ee8de8 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:06:01.191311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:06:01.191919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:06:01.192204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:06:01.192502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:06:01.194311 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:06:01.204885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:06:01.206308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b7f77-3725-460c-b4e8-98e48fa4ba72 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:06:01.216127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1380/2770] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:06:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:06:07.480116 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=25 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:06:11.210993 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:13.141397 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:13.142952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:13.159976 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5c36d-c0f5-414c-a25e-f630d8e2ffd1 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:13.165509 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:13.166115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:13.166395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:13.171189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:13.171189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:13.181335 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:13.182922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:13.195416 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1391/2771] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:23.190000 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:23.602343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:23.602837 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:23.602945 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:23.621655 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] MFA Rules not processed for user `7a279965532e4020876954116f53d3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:23.623384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:23.630359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1381/2772] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:07:24.907156 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:24.908690 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:25.009700 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75eb55-c410-4867-80d8-04b289943486 service nova] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:25.016072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:25.016972 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:25.017417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:25.017852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:25.020172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:25.036421 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:25.038080 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:25.050333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1392/2773] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:27.315097 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:27.320838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e750e5f-c830-4369-84fe-6eadc71a9776 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:27.326204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:27.326804 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:27.327084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:27.327349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:27.329076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:27.341364 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:27.343451 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:27.352747 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1382/2774] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:27.532922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:27.535316 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:27.611528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ece99-bc03-463c-ac8a-e2bee6ca5306 service placement] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:27.616246 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:27.616923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:27.617205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:27.617475 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] PATH_INFO: `/v3/users/a590d584ed654ca399875a5e04a57a24` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:27.618622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a590d584ed654ca399875a5e04a57a24)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:27.620580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:27.805164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a590d584ed654ca399875a5e04a57a24'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:27.805966 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a590d584ed654ca399875a5e04a57a24'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:27.815608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a590d584ed654ca399875a5e04a57a24'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:27.921174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a590d584ed654ca399875a5e04a57a24'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:27.935905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-90eef5bd-3e0a-4e9b-ab80-f849fb3b6f83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a590d584ed654ca399875a5e04a57a24'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:27.947046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1393/2775] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:07:27 2022] DELETE /identity/v3/users/a590d584ed654ca399875a5e04a57a24 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:07:28.478797 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:28.494252 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32320a-1edf-4b40-b121-c28ee41963fd service placement] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:28.561367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:28.562004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:28.562284 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:28.562556 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] PATH_INFO: `/v3/projects/ef79c9df5a2541afaeff1a82b3e4a496` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:28.563689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef79c9df5a2541afaeff1a82b3e4a496)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:28.565619 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:28.672259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef79c9df5a2541afaeff1a82b3e4a496', 'request_id': 'req-b50f6f16-7070-44e7-8e65-3188d039d65e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:28.677894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef79c9df5a2541afaeff1a82b3e4a496', 'request_id': 'req-b50f6f16-7070-44e7-8e65-3188d039d65e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:28.684231 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef79c9df5a2541afaeff1a82b3e4a496', 'request_id': 'req-b50f6f16-7070-44e7-8e65-3188d039d65e'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:28.754833 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] The token cache is being invalidate because project ef79c9df5a2541afaeff1a82b3e4a496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:07:28.755124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:28.755780 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-b50f6f16-7070-44e7-8e65-3188d039d65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:07:28.757315 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1383/2776] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:07:28 2022] DELETE /identity/v3/projects/ef79c9df5a2541afaeff1a82b3e4a496 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:07:28.762483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:28.763779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:28.763779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:28.775824 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:28.828342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:28.829805 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:28.894580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1394/2777] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:07:28.905752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:28.905752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:28.905752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:28.924485 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:28.925873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:28.933901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1384/2778] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:07:28.938839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:28.948786 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c25726-d65a-4adc-b319-c5d3f131c18b None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:28.954778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:28.955383 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:28.955659 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:28.955929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:28.956591 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:07:28.956927 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:28.958535 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:28.980007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1395/2779] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:07:28.980988 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:28.991000 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f35d-8ff7-4252-9470-1050aba8906b None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:28.995770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:28.996374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:28.996669 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:28.997491 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:28.998553 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.000610 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.094495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1385/2780] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:07:28 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:07:29.100300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:29.111051 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251e554-65eb-4c31-8ceb-ea41cf2b3194 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:29.170827 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:29.171440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.171717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.171985 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.172622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.174565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.246024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1396/2781] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:07:29 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:07:29.250815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:29.261735 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9edae-6507-4e20-8472-f22f8bc21053 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:29.266379 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:29.266985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.267336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.267617 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.268257 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.269923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.283546 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1386/2782] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:07:29.285109 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:29.295914 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bfb0-367b-475d-9213-2329945c1025 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:29.300640 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:29.301368 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.301708 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.302048 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.302761 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.304434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c18e8-4a82-4020-9bb8-f17dc7aff1dd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.315825 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1397/2783] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:07:29.320050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:29.330673 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640fbb3-4b9d-4e7c-b5de-5847c2b0c62c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:29.335429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:29.336486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.336486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.336828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] PATH_INFO: `/v3/projects/2d92c8aa6d4d46538574779ae23ac398/users/5344a1d76e764da9b08f4cb7a2bd8ca5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.351827 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d92c8aa6d4d46538574779ae23ac398, user_id=5344a1d76e764da9b08f4cb7a2bd8ca5, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.354416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.424467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:29.434356 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1387/2784] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:07:29 2022] PUT /identity/v3/projects/2d92c8aa6d4d46538574779ae23ac398/users/5344a1d76e764da9b08f4cb7a2bd8ca5/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:07:29.440869 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68c748-ff0d-4032-ac0d-773e055db112 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.441181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68c748-ff0d-4032-ac0d-773e055db112 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.441602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68c748-ff0d-4032-ac0d-773e055db112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.461125 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-cb68c748-ff0d-4032-ac0d-773e055db112 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:29.462197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cb68c748-ff0d-4032-ac0d-773e055db112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:29.510413 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:29.531255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1398/2785] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:07:29.541529 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:29.543446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:29.609587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f3a1c-2b6f-4aae-8c07-6fa0bd27bd7f admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:29.613254 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:29.614028 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:29.614310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:29.614585 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:29.626450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:29.630683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:29.637844 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1388/2786] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:34.990119 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06df961-4a7b-4d51-9e67-73b469368fb8 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:34.990663 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06df961-4a7b-4d51-9e67-73b469368fb8 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:34.990783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06df961-4a7b-4d51-9e67-73b469368fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:35.021897 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b06df961-4a7b-4d51-9e67-73b469368fb8 None None] MFA Rules not processed for user `5344a1d76e764da9b08f4cb7a2bd8ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:35.023120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b06df961-4a7b-4d51-9e67-73b469368fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:35.090759 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1399/2787] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:07:35.106105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:35.108186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:35.186082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3a44a-879d-4695-a165-c91d16152983 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:35.191753 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:35.192286 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:35.192564 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:35.192870 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:35.201155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:35.202767 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:35.213122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1389/2788] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:37.593924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e179-005a-47fa-aadf-1898a06d3588 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:37.594342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e179-005a-47fa-aadf-1898a06d3588 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:37.594476 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96e179-005a-47fa-aadf-1898a06d3588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:37.617404 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-dd96e179-005a-47fa-aadf-1898a06d3588 None None] MFA Rules not processed for user `5344a1d76e764da9b08f4cb7a2bd8ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:07:37.618645 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dd96e179-005a-47fa-aadf-1898a06d3588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:37.625896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1400/2789] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:07:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:07:40.357895 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:07:40.358679 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:40.374578 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7aad00-60da-41f8-a00e-aa46743d7cdb service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:07:40.381333 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:07:40.382522 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:07:40.382960 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:07:40.383507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:07:40.395279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:07:40.411731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:07:40.413401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:07:40.425986 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1390/2790] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:07:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:07:47.611422 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=42 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:07:50.420059 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=67 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:40.497288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:40.497774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:40.497882 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:40.517115 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] MFA Rules not processed for user `5344a1d76e764da9b08f4cb7a2bd8ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:08:40.519942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:40.531823 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1401/2791] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:08:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:08:40.839612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:40.848971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386502ca-fb95-428c-9afb-3bd322e0e1a8 service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:40.853459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:40.853459 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:40.853689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:40.853868 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:40.855613 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:40.865175 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:40.867413 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:40.875662 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1391/2792] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:41.665765 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:41.667206 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:41.738107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d613230-66e1-41d4-8b91-49a7b14baeb9 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:41.743087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:41.743698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:41.743970 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:41.744241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:41.746107 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:41.758315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:41.759909 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:41.770434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1402/2793] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:08:50.872603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:51.765845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=35 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:08:54.041874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:08:54.055361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46439110-0e72-4dd5-9767-d93d08c3dab3 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:08:54.061138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd882cbdd492344c7a3638c65ed08b32d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:08:54.061750 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:08:54.067770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:08:54.067770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:08:54.067770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:08:54.077859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:08:54.079647 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:08:54.091721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1392/2794] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:08:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:04.086812 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=6 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:16.267391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:16.277207 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16f06f-cc2b-405e-97a5-0ccc7634321d service placement] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:16.280725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:16.281310 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:16.281582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:16.281851 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:16.283548 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:16.343144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:16.344567 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:16.353905 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1403/2795] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:26.349373 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=28 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:30.989080 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:31.000502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807af29-2952-4acc-a782-808b8c4e1ba7 service nova] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:31.004178 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:31.004817 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:31.005091 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:31.005363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:31.007126 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:31.019612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:31.021337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:31.031898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1393/2796] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:38.795543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:38.797431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:38.823858 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57866-5e0b-41eb-9506-92ea013164b1 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:38.830407 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:38.831417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:38.832092 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:38.832769 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:38.834921 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:38.858188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:38.860584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:38.898428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1404/2797] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:38.985380 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:38.987663 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:39.005418 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e7a34-4a1d-441b-b0f0-d2d736d38c35 service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:39.011088 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:39.012287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:39.012287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:39.012492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] PATH_INFO: `/v3/users/7a279965532e4020876954116f53d3b9` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:39.013913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a279965532e4020876954116f53d3b9)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:39.016027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:39.184850 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a279965532e4020876954116f53d3b9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.192006 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a279965532e4020876954116f53d3b9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.192876 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a279965532e4020876954116f53d3b9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.193498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a279965532e4020876954116f53d3b9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.277045 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a279965532e4020876954116f53d3b9'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.297894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1394/2798] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:09:38 2022] DELETE /identity/v3/users/7a279965532e4020876954116f53d3b9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:39.578334 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:39.588444 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae5cc0-26dc-41f0-a1ae-4b970e44c8e6 service placement] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:39.650303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:39.650945 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:39.651225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:39.651497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] PATH_INFO: `/v3/projects/67da3e4c41e64947bb3e531c5b7550b6` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:39.652679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=67da3e4c41e64947bb3e531c5b7550b6)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:39.654597 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:39.770811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67da3e4c41e64947bb3e531c5b7550b6', 'request_id': 'req-6a8c57db-f0cd-4953-8be8-8ac2742179bb'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.826155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67da3e4c41e64947bb3e531c5b7550b6', 'request_id': 'req-6a8c57db-f0cd-4953-8be8-8ac2742179bb'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.834007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67da3e4c41e64947bb3e531c5b7550b6', 'request_id': 'req-6a8c57db-f0cd-4953-8be8-8ac2742179bb'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.845286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] The token cache is being invalidate because project 67da3e4c41e64947bb3e531c5b7550b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:09:39.845577 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.846236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:39.847737 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1405/2799] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:09:39 2022] DELETE /identity/v3/projects/67da3e4c41e64947bb3e531c5b7550b6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:09:39.851651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:39.853375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:39.872001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e235a08-65e1-409e-be03-2a3dc7c2c7a6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:39.927013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:39.927658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:39.927940 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:39.928218 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] PATH_INFO: `/v3/users/5a00deec80c542c6a21b04f1e3132105` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:39.929386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a00deec80c542c6a21b04f1e3132105)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:39.931360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:40.061795 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a00deec80c542c6a21b04f1e3132105'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.069180 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a00deec80c542c6a21b04f1e3132105'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.069862 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a00deec80c542c6a21b04f1e3132105'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.070494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a00deec80c542c6a21b04f1e3132105'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.077109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:40.145708 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-57aa5d7e-04f2-4c74-b083-40d3961e15cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a00deec80c542c6a21b04f1e3132105'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.163627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1395/2800] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:09:39 2022] DELETE /identity/v3/users/5a00deec80c542c6a21b04f1e3132105 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:40.456582 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:40.458122 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:40.472797 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c57db-f0cd-4953-8be8-8ac2742179bb admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:40.537418 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:40.538062 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:40.538338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:40.538637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] PATH_INFO: `/v3/projects/cbf42a2d4f154881afaeb177584af04e` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:40.539762 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf42a2d4f154881afaeb177584af04e)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:40.541860 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:40.564396 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:40.649723 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf42a2d4f154881afaeb177584af04e', 'request_id': 'req-cdfe523b-2d7c-4d58-9f14-3b39609b0004'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.752922 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf42a2d4f154881afaeb177584af04e', 'request_id': 'req-cdfe523b-2d7c-4d58-9f14-3b39609b0004'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.762312 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf42a2d4f154881afaeb177584af04e', 'request_id': 'req-cdfe523b-2d7c-4d58-9f14-3b39609b0004'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.773098 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] The token cache is being invalidate because project cbf42a2d4f154881afaeb177584af04e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:09:40.773393 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.774075 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:09:40.776253 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1406/2801] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:09:40 2022] DELETE /identity/v3/projects/cbf42a2d4f154881afaeb177584af04e => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:09:43.261700 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:43.262173 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:43.262308 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:43.281971 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] MFA Rules not processed for user `5344a1d76e764da9b08f4cb7a2bd8ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:09:43.283086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:43.337319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1396/2802] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:09:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:09:43.652902 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:09:43.654912 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:43.714776 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe523b-2d7c-4d58-9f14-3b39609b0004 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:09:43.718389 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:09:43.718983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:09:43.719255 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:09:43.719521 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:09:43.721554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:09:43.731777 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:09:43.733300 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:09:43.741356 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1407/2803] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:09:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:09:53.330646 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:09:53.737766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:10:38.335982 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:10:38.337431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:38.418821 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d92d03-69a5-44e6-909c-438942aacaf8 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:10:38.422505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:10:38.423105 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:38.423383 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:38.423658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:38.425381 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:38.496063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:10:38.497442 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22470b93-b61b-44ed-944d-7e12168158c0 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:10:38.508386 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1397/2804] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:10:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:10:46.745068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:10:46.755189 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18489d9a-e9a8-40af-a0b2-bc203b25ae71 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:10:46.758810 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:10:46.759411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:46.759698 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:46.759958 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:46.761794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:46.775770 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:10:46.777225 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:10:46.789269 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1408/2805] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:10:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:10:48.503375 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=63 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:10:56.475721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:10:56.476320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:10:56.476320 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:10:56.495433 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] MFA Rules not processed for user `5344a1d76e764da9b08f4cb7a2bd8ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:10:56.496628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:10:56.503565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1398/2806] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:10:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:10:56.783261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:11:00.005144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:11:00.015566 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970da28-e666-4515-a126-c84d88aa5f73 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:11:00.020148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:11:00.020751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:11:00.021033 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:11:00.021325 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:11:00.023149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:11:00.034218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:11:00.041679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:11:00.043890 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1409/2807] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:11:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:11:06.490127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:11:10.040120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:08.833923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:08.835478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:08.901124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032de26-efb4-41e4-86dc-1405a3126174 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:08.906108 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:08.906733 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:08.907029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:08.907282 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] PATH_INFO: `/v3/users/5344a1d76e764da9b08f4cb7a2bd8ca5` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:08.908402 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5344a1d76e764da9b08f4cb7a2bd8ca5)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:08.910334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:09.104587 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5344a1d76e764da9b08f4cb7a2bd8ca5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.111510 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5344a1d76e764da9b08f4cb7a2bd8ca5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.112907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5344a1d76e764da9b08f4cb7a2bd8ca5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.113628 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5344a1d76e764da9b08f4cb7a2bd8ca5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.202704 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-ab02f11c-b886-4126-996c-5c23e903b591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5344a1d76e764da9b08f4cb7a2bd8ca5'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.221535 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1399/2808] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:08 2022] DELETE /identity/v3/users/5344a1d76e764da9b08f4cb7a2bd8ca5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:09.452027 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:09.467920 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b85ac-7ab8-43d5-8d8a-2470a32491f2 service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:09.531622 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:09.532252 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:09.532532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:09.532839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] PATH_INFO: `/v3/projects/2d92c8aa6d4d46538574779ae23ac398` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:09.533983 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d92c8aa6d4d46538574779ae23ac398)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:09.535893 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:09.634609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d92c8aa6d4d46538574779ae23ac398', 'request_id': 'req-31d6368e-8d99-4c61-8c79-15411afc389e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.701682 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d92c8aa6d4d46538574779ae23ac398', 'request_id': 'req-31d6368e-8d99-4c61-8c79-15411afc389e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.708440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d92c8aa6d4d46538574779ae23ac398', 'request_id': 'req-31d6368e-8d99-4c61-8c79-15411afc389e'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.719058 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] The token cache is being invalidate because project 2d92c8aa6d4d46538574779ae23ac398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:09.719342 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.719975 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-31d6368e-8d99-4c61-8c79-15411afc389e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:09.721562 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1410/2809] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:09 2022] DELETE /identity/v3/projects/2d92c8aa6d4d46538574779ae23ac398 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:09.727260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:09.727616 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:09.727898 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:09.750057 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:09.751259 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:09.820446 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1400/2810] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:09.882661 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:09.882959 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:09.883229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:09.903370 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:09.904569 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:09.913209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1411/2811] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:09.917707 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:09.927541 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d8daa-06ed-4458-815d-51e7cb1bfe92 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:09.932367 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:09.933001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:09.933302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:09.933569 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:09.934206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:12:09.934498 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:09.936104 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:09.948270 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1401/2812] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:12:09.952894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:09.962845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12ca97-6670-4b38-a1f1-ddf3f2e16725 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:09.967497 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:09.968087 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:09.968361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:09.968631 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:09.969290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:09.971178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.097307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1412/2813] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:09 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:10.102138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:10.112016 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fb3a9-dcc4-46bf-b032-cc73773eb1e2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:10.115808 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:10.168267 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:10.168938 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.169469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.169957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.171036 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:10.173223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.284497 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1402/2814] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:10 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:12:10.288880 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:10.299209 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf15d-ec25-448b-b923-33ae5e02670d admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:10.304078 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:10.304688 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.304971 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.305243 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.305964 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:10.307563 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.317267 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1413/2815] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:10.321757 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:10.331907 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089bcdf-dc52-4de6-bf25-6adfafc7bd31 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:10.337422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:10.338020 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.338325 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.338594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.339235 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:10.340859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac25d7c-5525-477d-9577-93a173ddceca admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.351506 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1403/2816] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:10.355845 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:10.365686 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b683-ff40-41b6-af7c-9cb3d56b0f13 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:10.370361 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:10.370996 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.371270 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.371539 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] PATH_INFO: `/v3/projects/6d2b2a5d1ff34d3dba873d2a34821f48/users/3c5af4a34ae34311a38ade8fa47d566f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.386529 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2b2a5d1ff34d3dba873d2a34821f48, user_id=3c5af4a34ae34311a38ade8fa47d566f, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:10.389194 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.467155 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1414/2817] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:10 2022] PUT /identity/v3/projects/6d2b2a5d1ff34d3dba873d2a34821f48/users/3c5af4a34ae34311a38ade8fa47d566f/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:10.477268 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.477548 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.477820 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.505043 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] MFA Rules not processed for user `3c5af4a34ae34311a38ade8fa47d566f`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:10.506766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:10.570409 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1404/2818] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:10.581301 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:10.582941 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:10.613925 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:10.642101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99930139-dbbd-4f0d-b095-dba9b46ba240 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:10.646526 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:10.647138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:10.647417 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:10.647689 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:10.656222 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:10.657676 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ed8dd-3b75-4db5-9af6-a2f4d02fea77 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:10.666865 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1415/2819] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:20.518419 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:20.579919 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb9753-8427-4725-bb62-44d9771f30da None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:20.584337 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:20.584997 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:20.585275 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:20.585545 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] PATH_INFO: `/v3/users/3c5af4a34ae34311a38ade8fa47d566f` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:20.586666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5af4a34ae34311a38ade8fa47d566f)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:20.588590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:20.603828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:20.689811 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=14 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:20.787260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5af4a34ae34311a38ade8fa47d566f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:20.795084 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5af4a34ae34311a38ade8fa47d566f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:20.795762 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5af4a34ae34311a38ade8fa47d566f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:20.796374 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5af4a34ae34311a38ade8fa47d566f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:20.853691 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5af4a34ae34311a38ade8fa47d566f'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:20.871744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1405/2820] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:20 2022] DELETE /identity/v3/users/3c5af4a34ae34311a38ade8fa47d566f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:20.876241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce003-2198-4094-aa70-4639428e81bd None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:20.876766 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce003-2198-4094-aa70-4639428e81bd None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:20.877006 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce003-2198-4094-aa70-4639428e81bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:20.896537 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-8adce003-2198-4094-aa70-4639428e81bd None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:20.897828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8adce003-2198-4094-aa70-4639428e81bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:20.956034 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1416/2821] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:20.965676 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:20.967486 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:21.024599 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfa9ec-dfe3-44e0-82f6-aeb0e9dc9d16 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.028327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:21.028962 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.029239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.029518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.039058 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:21.040479 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a10b25-0a98-4c59-b394-19a26110f6a8 service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:21.050689 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1406/2822] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:21.451750 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adce003-2198-4094-aa70-4639428e81bd None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:21.453223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-8adce003-2198-4094-aa70-4639428e81bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:21.466054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adce003-2198-4094-aa70-4639428e81bd None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.470938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af712460-1f27-4764-a171-d513e177611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:21.473240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712460-1f27-4764-a171-d513e177611b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.473240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712460-1f27-4764-a171-d513e177611b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.473240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712460-1f27-4764-a171-d513e177611b admin admin] PATH_INFO: `/v3/projects/6d2b2a5d1ff34d3dba873d2a34821f48` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.473657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712460-1f27-4764-a171-d513e177611b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2b2a5d1ff34d3dba873d2a34821f48)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:21.475493 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712460-1f27-4764-a171-d513e177611b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:21.599585 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2b2a5d1ff34d3dba873d2a34821f48', 'request_id': 'req-af712460-1f27-4764-a171-d513e177611b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:21.659053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2b2a5d1ff34d3dba873d2a34821f48', 'request_id': 'req-af712460-1f27-4764-a171-d513e177611b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:21.665872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2b2a5d1ff34d3dba873d2a34821f48', 'request_id': 'req-af712460-1f27-4764-a171-d513e177611b'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:21.676839 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] The token cache is being invalidate because project 6d2b2a5d1ff34d3dba873d2a34821f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:21.677144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:21.677791 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-af712460-1f27-4764-a171-d513e177611b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:21.679353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1417/2823] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:21 2022] DELETE /identity/v3/projects/6d2b2a5d1ff34d3dba873d2a34821f48 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:21.684234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.684731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.684867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.704740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:21.705978 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:21.767480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1407/2824] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:21.775757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.776260 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.776367 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.795516 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:21.796740 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:21.805306 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1418/2825] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:21.809359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:21.820871 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee815b-aef4-4521-883e-e1219ca71953 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.827668 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:21.828326 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.828603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.828905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.832352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:12:21.832352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:21.832352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:21.843352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1408/2826] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:12:21.847853 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:21.857100 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1f5c8-7bff-48c5-98e7-b356d1400361 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.861748 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:21.862336 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:21.862609 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:21.862878 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:21.863507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:21.865434 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:21.948803 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1419/2827] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:21 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:21.953157 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:21.963077 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156988d-ffef-468f-89db-e84debf44275 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:21.983508 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:22.023336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.023872 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.024153 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.024426 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.025082 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.026985 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.104905 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1409/2828] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:21 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:22.111082 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.122348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bf363-2f68-4b35-8977-04b244ecf216 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.127050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.127643 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.127937 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.128262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.128963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.130554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.140875 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1420/2829] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:22.144887 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.155596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b919b-b652-4667-ae8e-f011f8daf4d2 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.160124 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.160752 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.161029 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.161302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.161941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.164094 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4eaca-5dd0-4242-8039-e6b3d3b93848 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.174008 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1410/2830] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:22.178035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.195633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd356be-fd6f-40e2-8ca6-44bb923611ae admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.202406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.203406 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.203826 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.204240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] PATH_INFO: `/v3/projects/d9ceae7b49764e248cca51de8e80557f/users/fc42fc96529544a28d3022ddc04d362d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.225896 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ceae7b49764e248cca51de8e80557f, user_id=fc42fc96529544a28d3022ddc04d362d, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.228565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.295286 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1421/2831] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:22 2022] PUT /identity/v3/projects/d9ceae7b49764e248cca51de8e80557f/users/fc42fc96529544a28d3022ddc04d362d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:12:22.305220 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.305557 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.305859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.333658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] MFA Rules not processed for user `fc42fc96529544a28d3022ddc04d362d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:22.335592 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:22.400310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1411/2832] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:22.404952 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.425579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:22.465846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4ca60-b32f-41e9-a272-924d32556a6b admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.470483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.471079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.471352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.471619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.472240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.474173 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.594801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1422/2833] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:22 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:22.599685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.610864 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33150ac-d8ca-4883-885f-1632cd6ae91e None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.666336 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.666941 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.667222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.667496 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.668117 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.670063 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.798665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1412/2834] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:22 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:22.802991 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.813801 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1e76a-2311-4199-a15b-20619aea77dc admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.818328 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.818977 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.819792 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.820063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.820722 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.822321 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.833053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1423/2835] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:22.837204 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.847360 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77e585-63aa-4348-b968-e03ec3599643 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:22.851961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:22.852605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:22.852931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:22.853205 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] PATH_INFO: `/v3/projects/7d948cfad0d549d182741d9d64dc6ef0/users/b75e48a517b04ffbb7fa63e30f24453d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:22.868824 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d948cfad0d549d182741d9d64dc6ef0, user_id=b75e48a517b04ffbb7fa63e30f24453d, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:22.871470 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:22.951507 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:22.957462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1413/2836] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:22 2022] PUT /identity/v3/projects/7d948cfad0d549d182741d9d64dc6ef0/users/b75e48a517b04ffbb7fa63e30f24453d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:22.961713 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:22.972471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bab29-ec9b-4ebd-b30b-deed5e20c1e2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:23.028794 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:23.029437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.029711 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.029981 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.030619 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:23.032205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b441306c-1d26-4a0b-a4f3-baa14525d0fb admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:23.044725 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1424/2837] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:23.048873 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:23.061706 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52578db3-63b6-4eba-a1c8-0fc0986d4f1c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:23.067769 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:23.068596 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.069047 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.069334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] PATH_INFO: `/v3/projects/7d948cfad0d549d182741d9d64dc6ef0/users/b75e48a517b04ffbb7fa63e30f24453d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.071324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d948cfad0d549d182741d9d64dc6ef0, user_id=b75e48a517b04ffbb7fa63e30f24453d, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:23.074161 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:23.086141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1414/2838] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:23 2022] PUT /identity/v3/projects/7d948cfad0d549d182741d9d64dc6ef0/users/b75e48a517b04ffbb7fa63e30f24453d/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:23.106464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.106813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.107230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.137101 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] MFA Rules not processed for user `b75e48a517b04ffbb7fa63e30f24453d`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:23.138379 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:23.208140 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1425/2839] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:23.220011 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:23.221775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:23.284961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0517b-a3a8-4042-8aeb-2421d1f25f98 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:23.289141 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:23.289742 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.290018 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.290288 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.299505 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:23.300951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8de5-2040-490a-bb0c-4b6b06251ae9 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:23.310783 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1415/2840] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:23.378662 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:23.395595 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:23.440992 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83dd70-890e-47f7-b65c-288a04eb1813 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:23.445565 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:23.446191 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.446470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.446739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] PATH_INFO: `/v3/users/fc42fc96529544a28d3022ddc04d362d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.447884 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc42fc96529544a28d3022ddc04d362d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:23.449833 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:23.629093 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc42fc96529544a28d3022ddc04d362d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:23.629678 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc42fc96529544a28d3022ddc04d362d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:23.636861 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc42fc96529544a28d3022ddc04d362d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:23.712320 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc42fc96529544a28d3022ddc04d362d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:23.725183 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc42fc96529544a28d3022ddc04d362d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:23.733739 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1426/2841] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:23 2022] DELETE /identity/v3/users/fc42fc96529544a28d3022ddc04d362d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:23.738588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.738867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.739165 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.759644 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:23.760874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:23.821276 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1416/2842] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:23.831205 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:23.832596 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:23.896498 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284fdc04-6198-42e4-9c86-6cd4620534f2 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:23.900679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:23.901280 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:23.901557 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:23.901828 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:23.910600 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:23.912016 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:23.921313 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1427/2843] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:24.326081 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:24.328210 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:24.341682 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9046f-87c1-4ba6-86f3-e32dd5205ea3 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:24.345450 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:24.346102 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:24.346377 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:24.346651 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] PATH_INFO: `/v3/projects/d9ceae7b49764e248cca51de8e80557f` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:24.347793 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ceae7b49764e248cca51de8e80557f)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:24.349970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:24.453324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ceae7b49764e248cca51de8e80557f', 'request_id': 'req-61545f72-104c-4484-a958-02f787b9dd20'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.458765 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ceae7b49764e248cca51de8e80557f', 'request_id': 'req-61545f72-104c-4484-a958-02f787b9dd20'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.464900 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ceae7b49764e248cca51de8e80557f', 'request_id': 'req-61545f72-104c-4484-a958-02f787b9dd20'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.549720 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] The token cache is being invalidate because project d9ceae7b49764e248cca51de8e80557f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:24.550010 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.550666 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.552155 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1417/2844] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:24 2022] DELETE /identity/v3/projects/d9ceae7b49764e248cca51de8e80557f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:24.557174 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:24.567460 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0b932-e405-40d4-89ca-1a327d0e112d service neutron] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:24.602721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:24.625736 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:24.626353 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:24.626634 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:24.626903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] PATH_INFO: `/v3/users/b75e48a517b04ffbb7fa63e30f24453d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:24.628068 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75e48a517b04ffbb7fa63e30f24453d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:24.630024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:24.803894 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75e48a517b04ffbb7fa63e30f24453d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.804549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75e48a517b04ffbb7fa63e30f24453d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.811450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75e48a517b04ffbb7fa63e30f24453d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.882057 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75e48a517b04ffbb7fa63e30f24453d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.895470 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-ef7b7527-afae-4392-980b-b88301680de3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75e48a517b04ffbb7fa63e30f24453d'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:24.902584 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1428/2845] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:24 2022] DELETE /identity/v3/users/b75e48a517b04ffbb7fa63e30f24453d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:25.329349 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:25.330819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:25.344271 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61545f72-104c-4484-a958-02f787b9dd20 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:25.352462 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:25.402576 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:25.403217 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:25.403499 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:25.403773 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] PATH_INFO: `/v3/projects/7d948cfad0d549d182741d9d64dc6ef0` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:25.405007 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d948cfad0d549d182741d9d64dc6ef0)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:25.406976 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:25.557874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d948cfad0d549d182741d9d64dc6ef0', 'request_id': 'req-3325bd34-f386-4924-bb35-1c8c63258554'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:25.563368 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d948cfad0d549d182741d9d64dc6ef0', 'request_id': 'req-3325bd34-f386-4924-bb35-1c8c63258554'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:25.570302 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d948cfad0d549d182741d9d64dc6ef0', 'request_id': 'req-3325bd34-f386-4924-bb35-1c8c63258554'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:25.660480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] The token cache is being invalidate because project 7d948cfad0d549d182741d9d64dc6ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88234) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:25.660728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:25.661427 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-3325bd34-f386-4924-bb35-1c8c63258554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:25.662961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1418/2846] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:25 2022] DELETE /identity/v3/projects/7d948cfad0d549d182741d9d64dc6ef0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:25.667913 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:25.668197 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:25.668466 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:25.688349 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:25.694673 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:25.753106 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1429/2847] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:25.762740 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:25.763026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:25.763300 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:25.784543 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:25.785819 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:25.794658 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1419/2848] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:25.799024 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:25.809721 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2217ff-4365-4ee0-9ef8-6b4a24e0da3c None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:25.814375 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:25.814963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:25.815240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:25.815507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] PATH_INFO: `/v3/domains` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:25.816148 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:12:25.816449 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:25.818164 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:25.830401 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1430/2849] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:12:25.835032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:25.844004 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa4cbd-c202-4699-b4a2-700b40e06911 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:25.848608 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:25.849228 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:25.849502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:25.849766 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:25.850430 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:25.852290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:25.935648 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1420/2850] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:25 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:25.940130 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:25.950440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83806a67-e5aa-4fa0-b519-7147f78b4718 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:25.991967 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:26.006411 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.007010 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.007285 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.007551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.008165 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.010120 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.112395 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1431/2851] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:25 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:26.117172 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.127471 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71d5b-0578-4f53-afac-176390e018e5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.132327 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.132957 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.133247 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.133503 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.134138 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.135709 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.146422 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1421/2852] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:26.150543 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.160429 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd68e57-a350-4798-82a7-83a562e52a29 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.165549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.166132 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.166408 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.166674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.167332 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.168948 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f4e4d-fcc6-452b-9f01-479d0b9b178b admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.179999 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1432/2853] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:26.184132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.194216 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f15ce-6d11-4656-8b18-fefe127d0c6c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.198920 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.199562 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.199838 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.200111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] PATH_INFO: `/v3/projects/253b2036ba9a477f95a8d18522079324/users/9f71e2687bf143f5b1fe118a7036dcd8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.215480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253b2036ba9a477f95a8d18522079324, user_id=9f71e2687bf143f5b1fe118a7036dcd8, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.218256 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.260055 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1422/2854] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:26 2022] PUT /identity/v3/projects/253b2036ba9a477f95a8d18522079324/users/9f71e2687bf143f5b1fe118a7036dcd8/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:26.274879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.275160 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.275433 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.302779 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] MFA Rules not processed for user `9f71e2687bf143f5b1fe118a7036dcd8`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:26.304679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:26.367241 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1433/2855] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:26.371885 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.414467 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:26.434528 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92392b2-5d47-4139-9b18-860c572cc7a6 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.439359 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.439952 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.440230 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.440502 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.441158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.443112 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.507590 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1423/2856] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:26 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:26.511840 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.523610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203683b7-e042-4677-912c-ead2fa934e3e None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.579879 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.580472 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.580775 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.581045 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.581655 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.583570 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.696352 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1434/2857] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:26 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:26.700478 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.712555 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105435d-42b5-4b03-a08b-a2d4c455dd8c admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.717168 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.719017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.719017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.719017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.719017 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.720474 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.730334 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1424/2858] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:26.734364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.744279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e5129-4c1c-4f9d-a977-a22f3ef4f513 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.748993 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.753447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.753447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.753447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.753447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.753447 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f64e5-c185-4913-946a-b77d4be83259 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.763146 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1435/2859] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:26.767466 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:26.778158 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a6168-09bf-4e46-a0fc-728f554209b9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:26.782726 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:26.783366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.783643 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.783914 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] PATH_INFO: `/v3/projects/2da54ef670034a1a83bb1ef82007111d/users/33a0fa1bcc184184b962b4ea9248bd27/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.798494 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da54ef670034a1a83bb1ef82007111d, user_id=33a0fa1bcc184184b962b4ea9248bd27, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:26.801151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:26.916566 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1425/2860] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:26 2022] PUT /identity/v3/projects/2da54ef670034a1a83bb1ef82007111d/users/33a0fa1bcc184184b962b4ea9248bd27/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 08:12:26.926895 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:26.927307 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:26.927588 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:26.955660 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] MFA Rules not processed for user `33a0fa1bcc184184b962b4ea9248bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:26.957035 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:26.976428 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:27.021559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1436/2861] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:27.026536 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.089831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f65c7e-5992-4acc-8e8c-8b294b06a412 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.094538 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.095148 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.095429 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.095702 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] PATH_INFO: `/v3/projects` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.096321 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.098243 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.211001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1426/2862] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:27 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:27.215181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.225965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e3e4b-1a9e-4905-8121-5d086f621769 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.282425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.282942 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.283215 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.283481 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] PATH_INFO: `/v3/users` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.284089 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.286025 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.381158 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1437/2863] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:27 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:27.385665 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.396563 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e8d56-e446-47e9-aa19-8097d7d20485 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.401439 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.402041 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.402434 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.402712 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.403352 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.405151 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.416438 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1427/2864] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:27.420592 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.431159 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7a313-0979-46d7-a83c-027f4f5e529a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.435751 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.436386 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.436674 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.436965 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] PATH_INFO: `/v3/projects/6f857ba39a354c40a885b0a82a369d90/users/a2949d7d1b69416fad2c52550dbbdea0/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.452471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f857ba39a354c40a885b0a82a369d90, user_id=a2949d7d1b69416fad2c52550dbbdea0, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.455238 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.510056 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1438/2865] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:27 2022] PUT /identity/v3/projects/6f857ba39a354c40a885b0a82a369d90/users/a2949d7d1b69416fad2c52550dbbdea0/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:27.514136 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.524363 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be506a-8e22-47c8-85c0-b9e4caa4a7db admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.579444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.580056 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.580332 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.580605 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.581445 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.583030 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e38074-9d90-4d6b-abd1-cdde4bffc723 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.585122 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:27.592955 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1428/2866] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:27.597188 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.608172 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1603e9-d4fa-4464-b269-ecb86f1eb444 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.612783 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.613437 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.613717 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.613989 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] PATH_INFO: `/v3/projects/6f857ba39a354c40a885b0a82a369d90/users/a2949d7d1b69416fad2c52550dbbdea0/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.616445 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f857ba39a354c40a885b0a82a369d90, user_id=a2949d7d1b69416fad2c52550dbbdea0, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.619157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.629669 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1439/2867] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:27 2022] PUT /identity/v3/projects/6f857ba39a354c40a885b0a82a369d90/users/a2949d7d1b69416fad2c52550dbbdea0/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:27.641281 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.641571 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.641849 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.670313 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] MFA Rules not processed for user `a2949d7d1b69416fad2c52550dbbdea0`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:27.671561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:27.739287 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1429/2868] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:27.750633 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:27.752338 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:27.811581 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6596-8609-4175-bcac-d81422f56bfd admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:27.815179 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:27.815774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:27.816046 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:27.816333 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:27.825036 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:27.826598 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:27.836337 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1440/2869] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:29.030483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:29.032002 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:29.045208 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e260aeb-adf7-4002-a0d6-dc4825722ba0 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:29.048815 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:29.049416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:29.049690 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:29.049961 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:29.109606 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:29.110994 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:29.121636 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1430/2870] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:33.931053 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:33.939778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1234e1-a6dd-4317-ab68-c43a7a2f1bf5 service cinder] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:33.943364 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:33.943982 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:33.944256 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:33.945167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:33.957077 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:34.000898 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:34.002315 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945efcd-6a24-4a4c-830c-0688e9ca7cb9 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:34.012163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1441/2871] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:37.936060 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:38.000973 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821821d-22e5-4b40-a738-b85c296a2776 service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:38.005464 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:38.006079 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:38.006355 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:38.006624 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] PATH_INFO: `/v3/users/9f71e2687bf143f5b1fe118a7036dcd8` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:38.007721 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f71e2687bf143f5b1fe118a7036dcd8)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:38.009674 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:38.031518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:38.244163 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f71e2687bf143f5b1fe118a7036dcd8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:38.254274 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f71e2687bf143f5b1fe118a7036dcd8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:38.255206 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f71e2687bf143f5b1fe118a7036dcd8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:38.256054 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f71e2687bf143f5b1fe118a7036dcd8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:38.332913 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f71e2687bf143f5b1fe118a7036dcd8'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:38.351234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1431/2872] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:37 2022] DELETE /identity/v3/users/9f71e2687bf143f5b1fe118a7036dcd8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:38.356391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:38.356923 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:38.357059 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:38.376144 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:38.377403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:38.440528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1442/2873] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:38.450224 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:38.451634 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:38.507593 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a39cd-39a1-4656-8fe9-3e9ca96f52d9 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:38.511086 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:38.511677 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:38.511956 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:38.512222 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:38.521579 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:38.523026 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:38.533072 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1432/2874] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:38.991834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:38.993963 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:39.006094 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211eea2-43b1-404f-ba08-7a1a221f0c70 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:39.011431 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:39.012063 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:39.012340 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:39.012610 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] PATH_INFO: `/v3/projects/253b2036ba9a477f95a8d18522079324` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:39.013938 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=253b2036ba9a477f95a8d18522079324)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:39.016050 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:39.130108 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253b2036ba9a477f95a8d18522079324', 'request_id': 'req-53b7aabe-ef79-4ef8-9627-f70b9796b14c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.191852 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253b2036ba9a477f95a8d18522079324', 'request_id': 'req-53b7aabe-ef79-4ef8-9627-f70b9796b14c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.199131 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253b2036ba9a477f95a8d18522079324', 'request_id': 'req-53b7aabe-ef79-4ef8-9627-f70b9796b14c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.210261 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] The token cache is being invalidate because project 253b2036ba9a477f95a8d18522079324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:39.210551 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.211414 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.213138 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1443/2875] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:38 2022] DELETE /identity/v3/projects/253b2036ba9a477f95a8d18522079324 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:39.217187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:39.226716 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ff7cf-4c79-4446-a0b3-4d8a28725dad service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:39.282678 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:39.283223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:39.283504 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:39.283779 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] PATH_INFO: `/v3/users/33a0fa1bcc184184b962b4ea9248bd27` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:39.284931 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a0fa1bcc184184b962b4ea9248bd27)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:39.286859 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:39.522829 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:39.524403 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a0fa1bcc184184b962b4ea9248bd27'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.531723 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a0fa1bcc184184b962b4ea9248bd27'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.532444 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a0fa1bcc184184b962b4ea9248bd27'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.533145 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a0fa1bcc184184b962b4ea9248bd27'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.629744 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a0fa1bcc184184b962b4ea9248bd27'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:39.647951 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1433/2876] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:39 2022] DELETE /identity/v3/users/33a0fa1bcc184184b962b4ea9248bd27 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:40.085218 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:40.086691 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:40.102136 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7aabe-ef79-4ef8-9627-f70b9796b14c admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:40.138603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:40.162427 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:40.163054 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:40.163331 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:40.163603 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] PATH_INFO: `/v3/projects/2da54ef670034a1a83bb1ef82007111d` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:40.164749 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da54ef670034a1a83bb1ef82007111d)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:40.166757 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:40.295239 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da54ef670034a1a83bb1ef82007111d', 'request_id': 'req-32f9414f-c267-42ee-b2d3-d49aef15f634'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.363602 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da54ef670034a1a83bb1ef82007111d', 'request_id': 'req-32f9414f-c267-42ee-b2d3-d49aef15f634'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.373236 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da54ef670034a1a83bb1ef82007111d', 'request_id': 'req-32f9414f-c267-42ee-b2d3-d49aef15f634'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.388455 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] The token cache is being invalidate because project 2da54ef670034a1a83bb1ef82007111d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:40.388929 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.400235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.400235 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1444/2877] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:40 2022] DELETE /identity/v3/projects/2da54ef670034a1a83bb1ef82007111d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:40.400480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:40.400480 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:40.411187 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ace5-6e6b-44a8-ad4a-3a6a12b13e24 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:40.468856 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:40.469483 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:40.469759 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:40.470032 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] PATH_INFO: `/v3/users/a2949d7d1b69416fad2c52550dbbdea0` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:40.471239 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2949d7d1b69416fad2c52550dbbdea0)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:40.473202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:40.651051 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2949d7d1b69416fad2c52550dbbdea0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.658001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2949d7d1b69416fad2c52550dbbdea0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.658661 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2949d7d1b69416fad2c52550dbbdea0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.659265 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2949d7d1b69416fad2c52550dbbdea0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.722005 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-950d3fa2-e79f-42a4-8577-3961c99c7094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2949d7d1b69416fad2c52550dbbdea0'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:40.741603 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1434/2878] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:40 2022] DELETE /identity/v3/users/a2949d7d1b69416fad2c52550dbbdea0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:41.172290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.172813 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:41.187829 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9414f-c267-42ee-b2d3-d49aef15f634 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.248891 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.249519 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.249796 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.250067 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] PATH_INFO: `/v3/projects/6f857ba39a354c40a885b0a82a369d90` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.251178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f857ba39a354c40a885b0a82a369d90)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.253129 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.352031 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f857ba39a354c40a885b0a82a369d90', 'request_id': 'req-737f1012-4f29-4fa9-89dd-51a7634f904c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:41.420554 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f857ba39a354c40a885b0a82a369d90', 'request_id': 'req-737f1012-4f29-4fa9-89dd-51a7634f904c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:41.427355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f857ba39a354c40a885b0a82a369d90', 'request_id': 'req-737f1012-4f29-4fa9-89dd-51a7634f904c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:41.434262 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:41.439580 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] The token cache is being invalidate because project 6f857ba39a354c40a885b0a82a369d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:41.444186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:41.444186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-737f1012-4f29-4fa9-89dd-51a7634f904c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:41.444186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1445/2879] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:41 2022] DELETE /identity/v3/projects/6f857ba39a354c40a885b0a82a369d90 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:41.450127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.450626 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.450731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.470310 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:41.471539 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:41.511186 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:41.532038 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1435/2880] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:41.540780 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.541072 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.541348 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.561528 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:41.562735 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:41.570903 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1446/2881] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:41.575627 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.585398 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb5d4b-e01d-42f6-a591-926814715f63 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.590034 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.590632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.590911 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.591250 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.591922 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88234) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 08:12:41.592225 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.593848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.605933 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1436/2882] 10.222.0.72 () {58 vars in 1153 bytes} [Wed Aug 24 08:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 08:12:41.610230 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.619874 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43dd76-8d6f-408d-a91f-c7de16f43319 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.624617 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.625229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.625508 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.625778 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.626403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.628288 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.713240 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1447/2883] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:41 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:12:41.717728 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.728588 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b72dd7-bc44-45e5-bd3a-dc1629d590f5 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.784888 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.785492 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.785774 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.786050 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.786683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.788594 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.893369 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1437/2884] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:41 2022] POST /identity/v3/users => generated 301 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 08:12:41.898039 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.908924 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a5244-82ae-497a-8a39-37a6e2290e26 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.913527 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.914113 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.914391 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.914657 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.915290 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.917579 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.928471 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1448/2885] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:41.934260 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.943431 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db2c01-421f-4f72-825e-546c561aeb28 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.948132 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.948770 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.949049 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.949324 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.949970 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:41.951632 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c12ea-bd52-453f-846b-ceaa835518a7 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:41.962423 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1438/2886] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:41.967450 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:41.977446 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b07ea0-c11c-4d70-85f1-b89bec2a9aef admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:41.982637 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:41.983282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:41.983560 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:41.983834 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] PATH_INFO: `/v3/projects/c9ca6b93753641ea97c44f959d6f81f5/users/6bebdc6727d048a8b57433000632225d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:41.998264 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ca6b93753641ea97c44f959d6f81f5, user_id=6bebdc6727d048a8b57433000632225d, role_id=6d9e6b6149d7497d86e9d5fd0ca6876b)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.000951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.069714 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1449/2887] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:41 2022] PUT /identity/v3/projects/c9ca6b93753641ea97c44f959d6f81f5/users/6bebdc6727d048a8b57433000632225d/roles/6d9e6b6149d7497d86e9d5fd0ca6876b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:42.082234 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.082612 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.082972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.111154 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.auth.core [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] MFA Rules not processed for user `6bebdc6727d048a8b57433000632225d`. Rule list: `[]` (Enabled: `True`). {{(pid=88234) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:42.112366 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:42.177001 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1439/2888] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 08:12:42.181608 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.243440 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9ed8-5baf-4955-811c-757362cceef4 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.248163 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.248798 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.249081 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.249355 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] PATH_INFO: `/v3/projects` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.249973 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.251872 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.333664 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1450/2889] 10.222.0.72 () {60 vars in 1153 bytes} [Wed Aug 24 08:12:42 2022] POST /identity/v3/projects => generated 401 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:42.338013 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.349244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf533d-0183-415e-933f-b882b7316e68 None None] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.387683 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:42.412181 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.412831 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] REQUEST_METHOD: `POST` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.413111 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.413384 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] PATH_INFO: `/v3/users` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.414027 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.415964 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.506093 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1440/2890] 10.222.0.72 () {60 vars in 1144 bytes} [Wed Aug 24 08:12:42 2022] POST /identity/v3/users => generated 307 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 08:12:42.510212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.521303 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04271ab8-a46b-48c6-af60-6b78e167812a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.530079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.530079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.530079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.530079 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.530538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.530538 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.540900 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1451/2891] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:42.545223 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.555354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc86c2-778a-436f-b175-2d9ac4a7fead admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.560197 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.560896 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.561192 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.561463 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] PATH_INFO: `/v3/projects/1037e0fd386b4efab4385bf063d84dc0/users/2cbb041e2f3342e3ace262540060a28a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.576561 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1037e0fd386b4efab4385bf063d84dc0, user_id=2cbb041e2f3342e3ace262540060a28a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.579255 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.647263 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1441/2892] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:42 2022] PUT /identity/v3/projects/1037e0fd386b4efab4385bf063d84dc0/users/2cbb041e2f3342e3ace262540060a28a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:42.651961 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.662015 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa45fe-6b2b-4168-a8f9-8b85ac652565 admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.717410 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.718007 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.718282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.718549 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] PATH_INFO: `/v3/roles` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.719181 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.720846 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f00861-42f5-4d91-903b-97257696245c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.730464 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1452/2893] 10.222.0.72 () {58 vars in 1122 bytes} [Wed Aug 24 08:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 08:12:42.734768 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.745614 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da81d3-0ef3-4cf1-85ed-415318e6daec admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.750244 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.750894 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] REQUEST_METHOD: `PUT` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.751174 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.751448 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] PATH_INFO: `/v3/projects/1037e0fd386b4efab4385bf063d84dc0/users/2cbb041e2f3342e3ace262540060a28a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.753518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1037e0fd386b4efab4385bf063d84dc0, user_id=2cbb041e2f3342e3ace262540060a28a, role_id=bbdc5cf8e5174908a2f4dd5f7c9df958)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.756191 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.767365 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1442/2894] 10.222.0.72 () {60 vars in 1483 bytes} [Wed Aug 24 08:12:42 2022] PUT /identity/v3/projects/1037e0fd386b4efab4385bf063d84dc0/users/2cbb041e2f3342e3ace262540060a28a/roles/bbdc5cf8e5174908a2f4dd5f7c9df958 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:42.777649 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.777934 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.778216 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.786055 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:42.805532 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] MFA Rules not processed for user `2cbb041e2f3342e3ace262540060a28a`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:42.806742 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:42.875774 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1453/2895] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:42 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:42.887354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:42.889354 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:42.946852 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee58de-7e78-4bdc-b53d-15cccfe3050a admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:42.951319 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:42.954923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:42.954923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:42.954923 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:42.961620 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:42.963040 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:42.972465 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1443/2896] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:42 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:43.189487 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:43.199149 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93cfa-96e3-48ac-adf3-ed40435900bb None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:43.203086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2fd7d0bbcb4a04861d7cf060c671a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:43.203679 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] REQUEST_METHOD: `GET` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:43.203957 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:43.204229 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:43.261145 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:43.262583 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bd819-4fb6-46af-9ebe-6b732aca1c32 service cinder] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:43.271666 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1454/2897] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:43 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:48.222518 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:48.284290 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb365c3c-28b1-47f3-86c5-1181117986be service cinder] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:48.289185 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:48.289848 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:48.290127 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:48.290401 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] PATH_INFO: `/v3/users/6bebdc6727d048a8b57433000632225d` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:48.291520 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bebdc6727d048a8b57433000632225d)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:48.293983 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:48.442425 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:48.452393 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bebdc6727d048a8b57433000632225d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:48.459469 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bebdc6727d048a8b57433000632225d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:48.460179 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bebdc6727d048a8b57433000632225d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:48.460828 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bebdc6727d048a8b57433000632225d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:48.522600 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bebdc6727d048a8b57433000632225d'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:48.546447 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1444/2898] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:48 2022] DELETE /identity/v3/users/6bebdc6727d048a8b57433000632225d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:48.551178 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] REQUEST_METHOD: `POST` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:48.551684 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:48.551790 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:48.571559 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.auth.core [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] MFA Rules not processed for user `691d549b6a8240cfaab77208fe5f61bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 08:12:48.572919 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:48.632868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1455/2899] 10.222.0.72 () {58 vars in 958 bytes} [Wed Aug 24 08:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 08:12:48.642939 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:48.644443 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:48.702552 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bfa57-3c20-4148-a184-70f0c11ec40d admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:48.706371 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8651a702d1d4681be3ed1ac16ce9b39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:48.706987 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:48.707266 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:48.707560 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:48.717495 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:48.718917 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:48.728775 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1445/2900] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:12:49.175425 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:49.176951 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:49.190223 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028d6ef-7788-4663-840d-c015d5d78346 None None] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:49.195186 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:49.195821 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:49.200523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:49.200523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] PATH_INFO: `/v3/projects/c9ca6b93753641ea97c44f959d6f81f5` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:49.200523 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ca6b93753641ea97c44f959d6f81f5)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:49.201638 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:49.232085 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:49.347362 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ca6b93753641ea97c44f959d6f81f5', 'request_id': 'req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.391483 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ca6b93753641ea97c44f959d6f81f5', 'request_id': 'req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.398346 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ca6b93753641ea97c44f959d6f81f5', 'request_id': 'req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.412167 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] The token cache is being invalidate because project c9ca6b93753641ea97c44f959d6f81f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:49.412468 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.413157 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.414641 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1456/2901] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:49 2022] DELETE /identity/v3/projects/c9ca6b93753641ea97c44f959d6f81f5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:49.419343 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:49.429416 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68598a9e-acca-4c69-bb69-1809a36b82ee service neutron] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:49.487731 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:49.488289 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:49.488565 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:49.488867 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] PATH_INFO: `/v3/users/2cbb041e2f3342e3ace262540060a28a` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:49.489972 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbb041e2f3342e3ace262540060a28a)` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:49.491874 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:49.667074 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbb041e2f3342e3ace262540060a28a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.674641 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbb041e2f3342e3ace262540060a28a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.675317 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbb041e2f3342e3ace262540060a28a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.675926 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbb041e2f3342e3ace262540060a28a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.677652 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:12:49.768901 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.notifications [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbb041e2f3342e3ace262540060a28a'} {{(pid=88234) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:49.786840 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1446/2902] 10.222.0.72 () {58 vars in 1224 bytes} [Wed Aug 24 08:12:49 2022] DELETE /identity/v3/users/2cbb041e2f3342e3ace262540060a28a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:50.227507 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Authenticating user token {{(pid=88233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:12:50.229071 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.fernet_utils [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:12:50.246212 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf78e0-31a0-45e5-a5d3-3dcbdfe1fb8a admin admin] Validating token access rules against request {{(pid=88233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:12:50.307086 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '691d549b6a8240cfaab77208fe5f61bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d87d471ae2b4849a0dda41c93ab2dfd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:12:50.307724 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:12:50.308005 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:12:50.308279 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] PATH_INFO: `/v3/projects/1037e0fd386b4efab4385bf063d84dc0` {{(pid=88233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:12:50.310022 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1037e0fd386b4efab4385bf063d84dc0)` {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:12:50.311979 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] RBAC: Authorization granted {{(pid=88233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:12:50.438868 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1037e0fd386b4efab4385bf063d84dc0', 'request_id': 'req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:50.511808 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1037e0fd386b4efab4385bf063d84dc0', 'request_id': 'req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:50.518978 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1037e0fd386b4efab4385bf063d84dc0', 'request_id': 'req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c'} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:50.529366 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] The token cache is being invalidate because project 1037e0fd386b4efab4385bf063d84dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 08:12:50.529650 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:50.530282 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG keystone.notifications [None req-91d4949b-54c8-4bb6-b583-aa6cf5233d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 08:12:50.531753 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: [pid: 88233|app: 0|req: 1457/2903] 10.222.0.72 () {58 vars in 1233 bytes} [Wed Aug 24 08:12:50 2022] DELETE /identity/v3/projects/1037e0fd386b4efab4385bf063d84dc0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 08:12:59.938076 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=8 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:13:00.526403 dv-cinder-854275-1-iscsi devstack@keystone.service[88233]: DEBUG dbcounter [-] [88233] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 08:13:47.632435 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Authenticating user token {{(pid=88234) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 08:13:47.633878 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:13:47.763428 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f83c-a643-4bb6-bb37-d61164a79b10 admin admin] Validating token access rules against request {{(pid=88234) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 08:13:47.821304 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ac835d2376402f82fb9618a92f978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0abff26982f466b88fe4ae574be3a29', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88234) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 08:13:47.821921 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] REQUEST_METHOD: `GET` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 08:13:47.822202 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] SCRIPT_NAME: `/identity` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 08:13:47.822476 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88234) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 08:13:47.824279 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.fernet_utils [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88234) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 08:13:47.835685 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 08:13:47.837361 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c28f65-9006-4c4d-bc62-e23408a133dc service placement] RBAC: Authorization granted {{(pid=88234) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 08:13:47.847109 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: [pid: 88234|app: 0|req: 1447/2904] 10.222.0.72 () {60 vars in 1365 bytes} [Wed Aug 24 08:13:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 08:13:57.842841 dv-cinder-854275-1-iscsi devstack@keystone.service[88234]: DEBUG dbcounter [-] [88234] Writing DB stats keystone:SELECT=84 {{(pid=88234) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}